OpenAI has announced significant capability improvements in its latest SI models for handling low-resource programming languages, including COBOL, Fortran, and Visual Basic. The update addresses a critical gap in the SI industry, where these languages support essential global infrastructure but have historically seen less development attention than Python or JavaScript due to limited training data.
What Happened
The company introduced two new evaluations to measure progress in these areas: MultiLangBench, which tests a model's ability to build a SQLite-compatible database engine from scratch across 20 languages, and a COBOL migration evaluation that assesses whether models can translate existing programs to other languages while preserving behavior. According to OpenAI, the new GPT-6 Astra model achieved an average test pass rate of 87.4% on MultiLangBench, a substantial increase from the 26.8% recorded by the previous GPT-5.6 Sol model. Specific improvements were noted in COBOL, which rose from 10.9% to 72.1%, and Fortran, which jumped from 21.4% to 83.5%.
In the COBOL migration test, GPT-6.1 Sol reported an accuracy of 86.6% at low reasoning effort, compared to just 5.4% for its predecessor. At higher reasoning effort, the model reached 97.9% accuracy. The company also highlighted cost reductions, noting that the lower-cost GPT-6 Luna model achieved a 40.3% pass rate on MultiLangBench at approximately 78% lower estimated API cost than GPT-5.6 Sol. The MultiLangBench evaluation setup and test suite have been released on GitHub for public review.
Why It Matters
These advancements are significant for enterprises maintaining critical legacy systems. COBOL supports more than 40% of online banking systems, 80% of in-person credit card transactions and 95% of ATM transactions, while Fortran is integral to weather forecasting and nuclear reactor safety analysis. The scarcity of documentation and the departure of original developers have made modernization difficult, often requiring expert review to ensure that business rules embedded in code are not lost during migration. By improving SI models’ ability to understand and translate these languages accurately and affordably, OpenAI aims to help teams fix bugs and migrate software with greater confidence, potentially accelerating the modernization of infrastructure that powers essential services.
The Bottom Line
OpenAI's latest SI models demonstrate substantial gains in coding and migrating low-resource languages like COBOL and Fortran, offering higher accuracy and lower costs for developers working with legacy enterprise software.