OpenAI Solves Legacy Software Migration: MultiLangBench Lifts COBOL Pass Rate to 97.9%, Modernizing Enterprise Stacks

Background and the Problem

Over 250 billion lines of legacy software written in COBOL, Fortran, and Visual Basic continue to power global financial clearings, airline logistics, and administrative mainframes. An estimated 70% of daily commercial transactions in the United States still touch COBOL backend logic. However, enterprises face an unprecedented succession crisis as original maintainers retire. These environments are severely low-resource in public internet datasets, causing conventional coding LLMs to hallucinate on rigid record alignments and legacy memory structures. Replacing these systems manually carries prohibitive costs and immense operational risk. OpenAI officially unveiled its landmark research into low-resource language engineering alongside the MultiLangBench benchmark.

Architecture and How It Works

1. Cross-Language Semantic Reasoning Alignment:

• Leverages frontier reasoning capabilities in GPT-6 Astra and GPT-6.1 Sol to map high-level system logic into low-resource syntactic targets, bypassing reliance on superficial surface token statistics;
• Systematically aligns with historical paradigms—including fixed-column formatting, unstructured memory buffers, and legacy compiler conventions;

1. The MultiLangBench SQLite-Scale Challenge:

• Requires models to implement an end-to-end, SQLite-compatible database engine with ACID transactions and B-Tree indexing from scratch across 20 programming languages;

1. Open-Source Evaluation Harness:

• OpenAI is open-sourcing the entire MultiLangBench suite, test configurations, and validation harnesses on GitHub.

Benchmarks and Measured Results

According to OpenAI official telemetry and technical reports (Figure 1, Benchmark Disclosures):

• MultiLangBench Database Engine Pass Rates:
• GPT-6 Astra (Max Effort) achieved an average pass rate of 87.4%, a massive leap over GPT-5.6 Sol's 26.8%;
• Astra cleared over 83% test pass rates in 19 out of the 20 evaluated languages;
• Dramatic Gains in Legacy & Low-Resource Stacks:
• COBOL: Rose from 10.9% to 72.1%;
• Fortran: Rose from 21.4% to 83.5%;
• Visual Basic: Rose from 27.5% to 89.4%;
• OCaml: Rose from 25.6% to 85.8%;
• The agile GPT-6 Luna scored 40.3% at an estimated $0.77 per sample (compared to $3.48 for GPT-5.6 Sol);
• Enterprise COBOL Migration Benchmark:
• GPT-6.1 Sol scored 86.6% at low reasoning effort and 97.9% at high reasoning effort (compared to 5.4% for GPT-5.6 Sol);
• The 86.6% run cost 39% less compute than the baseline run.

Getting Started for Developers

• Technical Post: Accessible at openai.com/index/low-resource-programming-languages/;
• Enterprise Migration Strategy: For technical leads managing monolithic banking or government stacks, this milestone establishes that frontier reasoning models can reliably automate legacy system transpilation, reducing multi-year modernization initiatives into verifiable CI/CD test-driven pipelines.