Senior Software Engineer, AI Benchmarking

About this role

# Senior Software Engineer, AI Benchmarking

• Project/Division: SecureBio AI Team
• Start date: As soon as possible
• Status: Full-time
• Work location: Remote, with office locations in Cambridge, MA and Berkeley, CA
• Reports to: Head of Engineering

## The role

SecureBio is looking for a Senior Software Engineer to build, scale, and run biosecurity evaluations on frontier AI systems. The tools you build will directly inform decisions made by frontier AI companies, policymakers, and researchers to improve the safety of AI systems.

Our biosecurity evaluations are regularly used to audit state-of-the-art models for biological capabilities and dual-use biology risks prior to their public release. Our work has been publicly cited in model cards from OpenAI, Anthropic, and Google, and has informed safeguards on frontier models. We are a member of the NIST US AI Safety Consortium, and a partner to the European AI Office.

## Responsibilities

• Perform pre-release and post-release assessments of frontier and open-source AI models using SecureBio's suite of biosecurity evaluations, which surface and measure dual-use biology capabilities in AI systems.

- Build adapters for models, identify and troubleshoot unexpected agent behaviors, and ensure that our evaluations are methodologically rigorous.
- Build, scale, maintain, and continuously improve SecureBio's suite of biosecurity evaluations.

- Improve our evaluations to use state-of-the-art agent frameworks, jailbreaking methods, and elicitation strategies.
- Develop and improve internal tools that enable our researchers to run capability assessments at scale.
- Contribute to and maintain cloud infrastructure to run our evaluations and analyses.
- Contribute analyses to our public dashboard of trends in AI biology capabilities ([https://securebio.org/benchmarks/](https://securebio.org/benchmarks/)).

## Qualifications

### A strong candidate would have the following qualifications:

• A proven track record in softw...