AI Coding Live News — Real-Time 𝕏/Twitter Updates & Breakthroughs

👀62

🔥96

OpenAI Solves Legacy Software Migration: MultiLangBench Lifts COBOL Pass Rate to 97.9%, Modernizing Enterprise Stacks

OpenAI officially announced breakthroughs in building and migrating software written in low-resource and legacy languages (COBOL, Fortran, Visual Basic, OCaml), introducing the MultiLangBench evaluation suite. Tackling the trillions of dollars in critical banking and enterprise mainframe infrastructure, the benchmark challenges models to synthesize a SQLite-compatible database engine across 20 distinct languages. GPT-6 Astra boosted average test pass rates from 26.8% to 87.4%, with COBOL soaring from 10.9% to 72.1%. In enterprise COBOL migration tasks, GPT-6.1 Sol achieved a 97.9% success rate while cutting inference costs by 39%.

Key Takeaways

OpenAI unveiled breakthroughs in low-resource and legacy languages with MultiLangBench, challenging models to implement database engines across 20 languages
GPT-6 Astra raised the average pass rate from 26.8% to 87.4% (clearing >83% in 19 languages), with COBOL hitting 72.1% and Fortran 83.5%
In enterprise COBOL migration benchmarks, GPT-6.1 Sol scored 97.9% (vs 5.4% for GPT-5.6 Sol) while reducing execution costs by 39%
OpenAI confirmed open-sourcing the MultiLangBench test suite on GitHub to accelerate automated modernization of mainframe software

Read full story

Linked Models & Agents:

#OpenAI#MultiLangBench#COBOL#Legacy Migration#GPT-6 Astra#Software Engineering

OpenAI@OpenAI·Collected 16d ago

👀113

🔥88

View on 𝕏 ↗

OpenAI open-sources MultiLangBench and reports low-resource language gains: GPT-6 Astra averages 87.4% building SQLite across 20 languages, GPT-6.1 Sol reaches up to 97.9% on COBOL migration

On October 19 OpenAI said its latest models are much better at low-resource languages such as COBOL, Fortran, Visual Basic and OCaml, and open-sourced MultiLangBench, which asks a model to build a SQLite-compatible database engine from scratch in each of 20 languages. GPT-6 Astra averages an 87.4% test pass rate (GPT-5.6 Sol: 26.8%), above 83% in 19 of 20 languages, with COBOL hardest at 72.1%. On a separate COBOL migration eval requiring byte-for-byte output parity after porting to C++, Java, Kotlin, Python or Rust, GPT-6.1 Sol scores 86.6% at low effort (GPT-5.6 Sol: 5.4%) and 97.9% at higher effort. All figures are OpenAI's own.

Key Takeaways

MultiLangBench: GPT-6 Astra (max) averages 87.4% across 20 languages vs 26.8% for GPT-5.6 Sol, above 83% in 19 of them (OpenAI-reported).
Per language: COBOL 10.9% to 72.1%, OCaml 25.6% to 85.8%, Fortran 21.4% to 83.5%, Visual Basic 27.5% to 89.4%.
Budget tier: GPT-6 Luna averages 40.3% at about $0.77 per sample vs $3.48 for GPT-5.6 Sol, roughly 78% cheaper.
COBOL migration: GPT-6.1 Sol scores 86.6% at low effort (GPT-5.6 Sol 5.4%) at about 61% of the API cost; 97.9% at higher effort, near saturation.
The eval setup, test suite and per-language results are on GitHub openai/multilangbench; the COBOL migration eval is not public.

Read full story

Linked Models & Agents:

⚡ GPT-6 Astra ⚡ GPT-6.1 Sol ⚡ GPT-6 Luna ⚡ GPT-5.6 Sol 🌐 OpenAI

#COBOL#Fortran#MultiLangBench#Benchmark