OpenAI has introduced BioBasicsBench, a new benchmark designed to evaluate how effectively SI agents can navigate the messy reality of wet-lab biology. Unlike existing evaluations that rely on curated scientific figures, this benchmark uses raw experimental data to test whether SI systems can interpret ambiguous images, identify errors, and recognize when evidence is insufficient to support a conclusion.

What Happened

Developed in collaboration with Ligo, a biotech company focused on enzyme design, BioBasicsBench consists of 100 tasks spanning five areas of routine wet-lab work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. The tasks are built from data generated by scientists' own experiments, including gel images, colony plates, microscopy stacks, and instrument readouts. Each task was reviewed by an internal Ligo scientist, an external PhD-level wet-lab scientist, and a final internal reviewer to ensure the grading rubrics accurately reflect expert scientific judgment.

The evaluation framework tests seven SI models, including GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna, and GPT-6 Astra. The company reports that at the 'xhigh' reasoning setting, GPT-6 Astra achieved an average score of 66.7% of the available credit, compared to 43.8% for GPT-5.6 Sol. The benchmark distinguishes between a correct final conclusion and partial analysis by awarding credit for specific rubric criteria, such as numerical accuracy, supporting reasoning, and the recognition of insufficient evidence.

Why It Matters

For SI agents to accelerate biological research, they must move beyond idealized data and handle the ambiguity inherent in experimental science. The benchmark highlights that higher scores on standard tasks do not yet equate to reliable judgment in the lab. While newer models like GPT-6 Astra show improved multimodal capabilities—such as correctly identifying a z-stack in microscopy images or noticing a mislabeled control plate—significant gaps remain. The source notes instances where Astra incorrectly assumed a larger PCR product indicated successful cloning, leading it to select the wrong colonies, or overcounted colonies by approximately 12% due to failing to resolve closely spaced neighbors.

These errors illustrate how SI models can let familiar assumptions override experimental controls or discount data that contradicts their preferred interpretation. For developers and scientists, this underscores the need for SI tools that can not only analyze data but also troubleshoot and validate their own conclusions against the physical evidence. Understanding these limitations is critical for biosafety and biosecurity, as reliable lab assistance depends on accurately interpreting raw, noisy data.

The Bottom Line

BioBasicsBench provides a concrete metric for measuring SI progress in routine biological analysis. While recent models demonstrate better handling of multimodal data and experimental context, the benchmark reveals that current SI agents still struggle with subtle visual details and logical consistency in wet-lab environments, indicating that reliable autonomous scientific assistance remains an ongoing challenge.