Legacy tech is hard to kill

As companies rush breathlessly forward to understand and incorporate AI into their organizations, they are often held back by legacy tech. I was thinking about this recently when I saw the news that AOL is ending its dial-up service at the end of September. It's the end of an era, but it’s not the end of dial-up, just one of the better-known brands.

As new technology comes along, it’s easy to think that it just neatly replaces what came before, but it doesn’t work like that. It tends to get layered on top, producing a messy mix of generations, resulting in management and updating nightmares aplenty. When you throw in acquisitions, it gets even messier.

Add to that the notion of tech debt, which IBM defines as “the future costs associated with relying on shortcuts or suboptimal decisions made during software development.” It means you take shortcuts today because you need to get stuff done and you can’t wait for the optimal time to release your software. It’s a concept that tends to bite just about everyone, no matter how intentional you may be about avoiding it. 

When you combine legacy bloat with tech debt, the default state in most large organizations, you are going to have a hard time implementing the latest technology. It’s what has held back companies with the best of intentions when it comes to digital transformation over the last decade. It could be what's holding you back as you struggle to implement your AI projects today.

Why mainframes still thrive

If you doubt me, let’s go back to the earliest computers, the mainframes, which in their original incarnations used to take up entire rooms. Mainframes may seem like a vestige of computing history – they turned 60 last year – but they are in fact very much alive and well, and running some of the largest companies in the world. 

A 2020 report from IBM pegged mainframe usage at 71% of the Fortune 500 companies. That means most significant companies are using the oldest computer technology you can imagine, even if they may have updated these systems along the way. More recently, IBM reports that “43 of the world’s top 50 banks and 8 of the top 10 payment companies rely on mainframes.” A 2025 Forrester report concurs, stating that “66% of enterprise hardware decision-makers reported mainframe usage. Certain industries reported similar adoption: 70% for financial services and insurance and 62% for business services, transportation and construction as a group.”

That’s pretty wild, and something to keep in mind as you think about your AI readiness. How much of your data is packed away, say in an old COBOL program? If you're a hospital, insurance company, government or airline, chances are at least some data is trapped in these ancient systems.

Can AI help solve tech debt?

If you ask an AI enthusiast how to solve a difficult problem with AI, chances are they will tell you by using more AI. As we’ve talked about in this space many times, data is the golden ticket to AI success and garbage in/garbage out still applies. If your data is in good shape, your company is going to be in a much better position to build applications on top of large language models.

But if your data is trapped inside legacy systems, you’re going to face massive challenges due to the cost and complexity of getting the data into a format that makes it usable for today’s AI tools. Companies keep ancient systems in place, not because they are against modernizing, but more likely because these systems are crucial to their operations, and they don’t want to mess with them. ‘If it ain’t broke, don’t fix it’ is a maxim many companies live by.” But AI is a different animal and suddenly updating those systems takes on more urgency.

Thomas Squeo, CTO for the Americas at Thoughtworks, a digital consultancy, says many of these companies could be at a crossroads now. “AI has made that data lock-in an existential threat for some enterprises because they're going to be out-competed by new players that are not encumbered by legacy systems,” he said.

As he sees it, the data needs to be extracted from these older systems and put into a modern data store like Google BigQuery or Amazon Redshift where you can make better use of it for AI. He also points out that there are tools like Mulesoft, Boomi or other similar platforms to wrap the mainframes in APIs to make it easier to extract the data and move it into these cloud applications.

"AI has made that data lock-in an existential threat for some enterprises because they're going to be out-competed by new players that are not encumbered by legacy systems."

It turns out that the folks who want to throw AI at the problem may be right in this instance. A McKinsey report released in June cited a case by a large bank that used agents to update legacy applications, saving 50% over the older manual processes. “These squads [of agents] retroactively document the legacy application, write new code, review the code of other agents, and integrate code into features that are later tested by other agents prior to delivery of the end product,” the report stated.

One thing to keep in mind, however, is you may want to rethink your processes, as well. Mike Bechtel, chief futurist at Deloitte, suggests this is a good opportunity to explore new approaches. “Good does not come from making bad things faster,” Bechtel told me. “And what I mean by that is sometimes we're stuck with the systems we have rather than the systems we would have created from first principles,” he said.

There are probably far more legacy systems out there than you might imagine, some of which play a central role in how a company operates. If you’re going to invest the time to port these systems to make their data accessible to AI, you may also want to reconsider how those systems function because chances are, your business has changed substantially since they were first created. The challenge lies in figuring out how to bridge the gap between these older systems and modern innovation without doubling down on yesterday’s mistakes.

~Ron

Featured photo by NASA - Great Images in NASA Description, Public Domain on Wikimedia