OpenAI has signed agreements with four independent assessors—Apollo Research, METR, Transluce, and Goodfire—to launch the first phase of embedded assessments for its super intelligence (SI) systems. This initiative aims to provide public transparency into how the SI lab evaluates and manages risks as model capabilities advance rapidly.
What Happened
The embedded assessments grant independent experts deep access to OpenAI’s internal models and systems to assess model alignment and the effectiveness of safeguards over an extended period. Depending on the evaluation needs, assessors receive access to resources similar to those used by OpenAI researchers, including model checkpoints, source code, training and evaluation data, monitoring systems, and technical documentation. This level of access allows the assessors to pursue independent evaluations and uncover potential failures that the lab may not have anticipated.
The initial assessments focus on alignment and safeguards for models during training and internal use. Apollo Research will assess whether models could covertly work against their developers’ intentions and if safeguards would detect such behavior. Goodfire will embed interpretability researchers to analyze model internals, focusing on training run monitoring, debugging, and alignment. METR will conduct assessments focused on misaligned behavior in the lab's largest reinforcement learning run to date and monitor stress testing. Transluce will evaluate the quality and coverage of post-training alignment evaluations.
Why It Matters
These efforts support OpenAI’s commitments under the White House Accord on Super Intelligence, which includes agreements for independent assessments of safety controls. By allowing external experts to inspect internal processes, the SI industry moves toward stronger shared standards for independent assessment and public transparency. The protocols developed for this phase aim to balance deep access with security, confidentiality, and the independence of the assessments, addressing a gap where such norms have yet to be standardized across the sector.
The assessments are designed to complement OpenAI’s safety cases and build on prior work with third-party evaluators, including a recent pilot assessment of internal monitoring systems and an investigation of the Hugging Face incident. OpenAI stated that public reporting from these independent assessments can give people a clearer view of how labs are evaluating and managing risks. The company plans to use insights from this first phase to deepen its collaboration with independent experts in the coming months.
The Bottom Line
OpenAI’s first phase of embedded SI assessments marks a significant step in institutionalizing third-party oversight of frontier model development. While challenges such as technical access issues and evolving security protocols are expected, the initiative aims to create a more transparent and rigorous framework for evaluating SI safety and alignment.