Short answer: AI is good at reading old code and at the repetitive middle of a migration. The cases everyone quotes were large, mechanical changes with a compiler and tests to check every step, and people still finished the hard part. Deciding what an old system should do, and proving the new one does the same, remains engineering work.
People who search for this usually type a company name next to it: Morgan Stanley, Airbnb, Anthropic, IBM. The numbers attached to those names travel well. A year and a half of work done in six weeks. 4,500 developer-years saved. Nine million lines of legacy code.
So we went back to what each company actually published, to see what was done, what was measured and what was still left to people. Most of it holds up. Very little of it is what a company with a fifteen-year-old product has in mind when it says "modernize".
What did the famous cases actually do?
Airbnb, March 2025. About 3,500 React test files moved from Enzyme to React Testing Library in six weeks, a job Airbnb had estimated at a year and a half by hand. The first bulk run converted 75% of the files in four hours. Four days of tuning took it to 97%, and the last hundred or so were finished by hand.
Google, 2025. Over twelve months, three developers ran 39 migrations of ID types with an in-house model. It wrote 74% of the code changes and 69% of the individual edits, and the team estimated the total time at half of what it would have been. A separate paper describes 5,359 test files moved from JUnit 3 to JUnit 4 in three months.
Amazon, August 2024. Tens of thousands of production applications upgraded from Java 8 or 11 to Java 17 with Amazon Q Developer. AWS put the saving at 4,500 developer-years, and Andy Jassy wrote that 79% of the generated changes shipped without edits.
Morgan Stanley, 2025–2026. An in-house tool, DevGen.AI, reads legacy code (COBOL, Perl, languages of the bank's own) and writes plain-English specifications that developers use to rebuild it. By mid-2025 it had worked through nine million lines. In July 2026 the bank told CIO Dive the figure was past 17 million.
Anthropic, February 2026. A post saying Claude Code can automate the exploration and analysis that take up most of a COBOL modernization. IBM, which sells mainframes, closed 13% lower the same day.
Every one of these is a company describing its own tool or its own project. That isn't a reason to doubt them, but it does mean nobody outside checked the numbers.
What do those numbers measure?
Less than the headlines suggest, and it helps to read them one by one.
The time saved is an estimate against a manual job that never happened. Google's 50% is the engineers' own estimate. Amazon's 4,500 developer-years was worked out from the number of dependencies upgraded, at an assumed day or so of work each. Jassy's baseline, about 50 developer-days per application, is a typical figure too, not something measured beside the AI run.
Then the $260 million — often read as labour saved, which it isn't. AWS describes it as a yearly saving from the faster Java 17 code needing fewer servers.
"Lines" changes meaning between reports. Morgan Stanley's nine million were lines the tool had read and described, not lines rewritten and shipped. A year later the bank said "modernized" instead. AWS reports close to 8 billion lines analysed by its Transform service, which is a count of input, not output.
And a team's own sense of how much faster it went is weak evidence on its own. In METR's trial, experienced developers believed AI had sped them up by 20% when it had slowed them down. We went through that study in Is AI making developers faster?
What do the cases have in common?
Each was the same change made thousands of times. A test library, a Java version, an ID type. Nobody had to decide what the code should do; the old behaviour was the answer, and a compiler and an existing test suite said whether each attempt kept it.
That is the setting where models do well. Airbnb ran every file through checks and retries until it passed, and said what made the difference was choosing which related files to show the model, not the wording of the prompt.
The people are in the sources too, if you read past the headline. Airbnb finished the last files by hand, and a later talk put code review at two more weeks. In Google's JUnit migration, the bottleneck was how fast engineers could review the changes. At Amazon, one in five generated changes needed a person to fix it. Morgan Stanley's head of technology told the WSJ the tool doesn't write new code as well as a developer, so developers still write it.
None of these was "rewrite our billing system" — that's a different job.
Why is old code harder for AI than new code?
Because what the code means isn't all in the code.
A fifteen-year-old system has rules nobody wrote down, workarounds for a client that left in 2016, and behaviour other systems quietly depend on. A model can describe what a function does. It can't know which of those behaviours matter.
The measurements point the same way. METR's 2025 trial ran on mature open-source projects of over a million lines, and the developers there were slower with AI, not faster. An earlier study of code translation gave models 1,700 samples to move between languages, including two real projects. Between 2.1% and 47.3% came out correct, depending on the model, and the failures fell into 15 kinds of bug. Those were 2023's models, and newer ones do better; the kinds of bug are the part that hasn't aged.
So the honest summary is uneven. AI is strong at reading a large old codebase and explaining it, and at repeating a known change many times. It's weakest where legacy work is hardest: deciding, and proving.
Where does it still need an engineer?
Deciding what the system is supposed to do. Old code does a lot of things on purpose and a few by accident, and the model can't tell which is which. Somebody who knows the business has to, because a "bug" that every invoice since 2014 depends on is a feature now.
Proving the new version does the same as the old one. A translation that compiles and reads well is the easy half. The hard half is a test that fails when behaviour changes, run against real data, before anyone switches over.
The parts outside the code: data that has to move, jobs that run at night, the integration nobody documented, the cutover itself. These are where modernization projects usually go wrong, and none of them is a prompt.
How should a company with an old product start?
Smaller than the case studies suggest, and in this order. Anthropic's own COBOL post recommends much the same: one component with clear boundaries, documented first, then migrated step by step with tests at each step.
Have the model read before it writes
Point it at one module that hurts (slow, fragile, or the one nobody wants to touch) and ask for a written account: what it does, what it depends on, what calls it. Then have someone who knows the system correct that account. The corrections are the most valuable part, because they're the knowledge that lives only in people's heads.
Turn what you learned into tests
Michael Feathers called these characterization tests: tests that pin down what the code does today, right or wrong, so you notice when it changes. They're tedious to write by hand, and models are good at drafting them. A person still decides which behaviour is intended.
Replace it piece by piece
Martin Fowler's strangler fig pattern: build the new part beside the old one, send a slice of the traffic to it, and retire the old part when nothing uses it any more. It's slower on paper than a rewrite. It also keeps the business running the whole time, and every step can be undone.
Measure delivery, not lines
How long a change takes to reach production, how often releases fail, what the system costs to run. If those don't move, the modernization hasn't happened yet, however much code changed.
This is also how we approach these systems: read the old code with AI doing much of the reading, fix what hurts most, then renew it piece by piece while it keeps running.
Common questions
Can AI convert COBOL to Java?
It can translate it. Whether the result does the same thing is a separate question, and the vendors say so themselves. Anthropic's post leaves it to the team to decide whether the tests are enough, and AWS's mainframe service now produces requirements traced to the source, from which developers generate and check the new code.
Will AI replace a modernization project?
No, but it changes where the time goes. Reading and documenting the old system used to take months of a team's time, and that part shrinks. Deciding, testing, moving data and switching over don't.
Which tools do companies use?
The cases above used Amazon Q Developer and AWS Transform, Claude Code, and tools built in-house at Google and Morgan Stanley. The tool matters less than the setup around it: a fixed target, checks that run on every change, and a person who reviews the result.
How long does it take?
Nobody can say before reading the code, and the six weeks in Airbnb's case was one repeated change, not a system. A first module, read, pinned down with tests and replaced, is a fair unit to estimate from.
