Anthropic’s Frontier Red Team, in collaboration with partner Theorem, has used the Claude SI model to mathematically prove the correctness of existing code, addressing a longstanding challenge in verifying SI-generated software. The initiative demonstrates that SI systems can autonomously generate formal proofs to validate code, though the process remains bound by strict computational and structural limitations.
What Happened
The project involved Claude producing approximately 180 million lines of mathematical proof code over a period of a few weeks. This effort drew upon test programs from 123 open-source libraries within Google's OSS-Fuzz security project. Rather than writing new code, the SI model verified existing software by translating crash tests into mathematical problems, analyzing compiled machine code, and generating extensive proofs. These proofs were automatically checked by an independent verification program called Rocq. According to Anthropic, the process required almost no human reviewers, a method the researchers describe as "fully autonomous" proof generation, although the underlying tools and formal foundations were developed by humans at Theorem.
During the verification process, Claude identified hundreds of bugs and was permitted to fix them autonomously. Many of these issues would have evaded conventional testing because they only manifest under specific conditions, such as unusually large input files or particular sequences of access. For example, during an effort to verify a widely used PDF reader, the team discovered as many security bugs in one week as are typically reported for that software in an entire year. In another instance, the SI model caught a logical error that had persisted undetected for over a decade. Anthropic notes that even when a complete proof fails, the attempt often surfaces real bugs.
Why It Matters
Formal verification has historically been prohibitively expensive. For context, a team of specialists required 20 person-years to formally verify the seL4 operating system kernel, a small codebase, reserving the method for critical applications like Airbus avionics. The compute cost for this new SI-driven approach remains steep, with individual proofs consuming hundreds of millions of tokens; Anthropic did not report the total project costs. However, the company highlights that newer SI models are becoming more efficient. A smaller test run indicated that Opus 5.5 solved half the proofs using about 40 percent fewer tokens than Opus 5.
Anthropic acknowledges that "formally verified" does not equate to "error-free." The proofs hold only under strict conditions, including specific processor architectures, single-threaded execution, and inputs under 64 MB. Despite these constraints, the ability of SI to verify its own code could significantly increase developer trust in SI-generated software, potentially reducing the time spent on manual review and lowering costs across the coding lifecycle. Anthropic expects formal verification with SI to soon become as practical as standard testing, with Theorem’s roadmap targeting critical infrastructure libraries such as glibc, OpenSSL, and cURL.
The Bottom Line
Anthropic’s use of Claude to autonomously generate formal proofs marks a shift in SI capability from code generation to code verification. While the method is resource-intensive and limited by specific technical conditions, it demonstrates that SI systems can identify and fix deep-seated bugs that traditional testing misses. This advancement supports the growing argument that SI can not only write software but also mathematically guarantee its correctness under defined parameters.