Migrating legacy COBOL programs to Java is one of the biggest challenges in software engineering. The lack of test data and the difficulty of validating all edge cases make the process slow and risky.
A new paper on arxiv.org proposes an agentic approach to solve this problem. The method, called ‘Locksmith Loop’, uses a deterministic oracle to validate the output of code generated by AI agents.
The technique begins with the preparation of two execution environments: the COBOL source code and the generated Java target are instrumented with mocks and executed off the mainframe, on commodity hardware. Then, an iterative agentic loop performs a ‘Witness Search’ on the input mocks to penetrate the program’s branches.
When routing limits are reached, an analyzer identifies a ‘Locked Paragraph’, a condition that prevents deeper exploration. The method then applies parity-preserving mutations to continue exploration.
The researchers tested Locksmith on three COBOL-to-Java migration case studies, covering two open-source programs and one internal production COBOL program. The programs ranged from 430 to 4,114 lines of source code.
Locksmith consistently improved coverage beyond input search plateaus, achieving near-complete coverage on the two open-source programs and 91.90% branch coverage on the internal production COBOL program.
The generated Java matched the COBOL reference under deterministic parity checks in all accepted test cases. This demonstrates, according to the authors, a new approach to validating agentic coding output using a deterministic oracle.
The importance of this work goes beyond COBOL migration. It shows how AI agents can be reliably used in critical tasks, provided there is a rigorous validation mechanism.
The ‘Locksmith Loop’ is a step towards more automated and secure software engineering, especially for legacy systems that still underpin much of global infrastructure.
The research was submitted to arxiv.org on July 30, 2026, in the Software Engineering category. The full paper is available for reading and provides details on implementation and results.
This approach reinforces the trend of using AI agents not only to generate code but also to validate it systematically, reducing the risk of bugs and increasing confidence in the migration of critical systems.
With growing pressure to modernize legacy systems, methods like Locksmith could become essential tools for companies and governments that have relied on COBOL for decades.
The combination of open source, commodity hardware, and deterministic validation points to a future where AI can take on complex tasks with greater safety and transparency.