OpenAI's models are now proving more effective in assisting the banking sector with software migration from legacy programming languages, particularly systems like COBOL, Fortran, and Visual Basic, which are still widely used. Approximately 40% of banks globally still rely on COBOL as their core system, and migrating to new languages can cost millions of dollars, with significant time and risk involved. As COBOL programmers gradually retire, completing the migration in the coming years has become an urgent task. OpenAI stated that improvements in the model's COBOL capabilities are guided by feedback from developers and enterprise clients; in practical applications, ensuring consistent system behavior post-migration hinges on surviving program behavior and boundary conditions for numerical computations. This blueprint indicates that AI models require long-term testing and auditing on top of existing automation tools to operate reliably in production environments.
Progress in Testing Model Behavior Retention and Migration Capabilities for Legacy Languages: Highlights and Limitations
OpenAI has tested the model's performance on tasks such as building SQLite-compatible database engines under specified languages using the MultiLangBench evaluation suite. The GPT-6 Astra achieved an average pass rate of 87.4% across 20 languages, a significant improvement over the previous generation GPT-5.6 Sol's 26.8%. However, for COBOL, the most challenging language, Astra's pass rate remains at only 72%. These figures indicate that while overall performance has improved, there is still a considerable distance to cover to meet the banking sector's stringent requirements while retaining existing data structures, financial rules, and fault tolerance mechanisms.
Limits of Accuracy and Behavior Retention in Cross-Language Migration: Production Suitability Under Review
OpenAI has also evaluated the migration of COBOL programs to modern languages such as C++, Java, Kotlin, Python, or Rust, while maintaining original behavior. These tests indicate that successful migration requires not only "code passing tests" but also the retention of key business rules, such as handling numerical rounding and missing records. Under high reasoning intensity, GPT-6.1 Sol achieved an accuracy of up to 97.9%, while under low reasoning intensity, it was 86.6%. OpenAI noted that to truly achieve production-level migration, all tests must pass; otherwise, additional expert reviews and systematic testing are necessary. These results show that AI-assisted migration still requires strict alignment with on-site business logic in risk-sensitive financial applications.
Background information indicates that the current model's applicability to legacy languages is limited by its depth of understanding of logical reasoning and data structures. Compared to general software development, banking core systems require predictable outputs and strict compliance, which cannot be resolved solely through automated generation models. OpenAI stated that in advancing modernization migration, integration and validation with clients' on-site testing processes, business rules engines, and data governance frameworks are still necessary to ensure that "all tests pass" without operational risks.
In summary, this round of research shows that AI technology has indeed made significant progress in legacy language migration, particularly in improving performance and pass rates in multilingual environments, but COBOL remains the biggest challenge. Governments and financial institutions looking to smoothly transition existing COBOL systems to modern architectures cannot rely solely on the model's automation capabilities; they also need supporting human reviews, existing rule definitions, and long-term monitoring systems. OpenAI pointed out that behavior retention capabilities provide practical value for teams maintaining and modernizing legacy systems, but rigorous testing and evaluation are still required before deployment in production environments to ensure compatibility with business rules and dependencies.