How to succeed at AI

If you were trying to read the AI tea leaves this week, you could be forgiven for having a hard time. There was no shortage of AI-related news and it was a series of increasingly mixed messages. As an example, we had OpenAI announcing $1 billion in revenue in July, a sign that the LLM company is gaining paying users and popularity — and that was even before the release of ChatGPT-5. That sounds pretty good, right?

Sure does, but does it mean that enterprise companies are actually using AI to successfully enhance their businesses? Maybe not. MIT NANDA released a study on the use of AI in the enterprise and concluded that, so far at least, the vast majority of companies are having a hard time building projects that provide meaningful ROI.

“Despite $30-$40 billion in enterprise investment into GenAI, this report [found] a surprising result in that 95% of organizations are getting zero return,” the report executive summary stated. “Just 5% of integrated AI pilots are extracting millions in value, while the vast majority remain stuck with no measurable P&L impact,” it went on.

If you’re curious about the methodology, the report looked at 300 publicly disclosed AI initiatives, while conducting “structured interviews” with folks from 52 organizations. It also conducted surveys with 153 senior leaders at four industry conferences, according to the report.

I can hear you saying, but hey, that’s kind of a limited group to be drawing such harsh conclusions, but it’s in line with a McKinsey report released in June. “Nearly eight in ten companies report using GenAI—yet just as many report no significant bottom-line impact. Think of it as the ‘GenAI paradox,’” the McKinsey report stated.

It’s enough to make any executive wonder if they’re just chasing the latest tech hype cycle, but it’s not all bad news by any means. It’s important to remember that there are ways to succeed at AI, but it’s going to require rethinking your processes, while preparing your employees for the big changes ahead.

Which companies succeed?

Both reports also made it clear that there are benefits to be had, it’s just hard for most companies to reap those benefits to this point. Everyone says it’s early days for AI and it’s true. Although it’s a technology that’s been around for decades, when it comes to generative AI, the kind that creates things, we are still less than three years from the release of ChatGPT. 

Interestingly, the MIT NANDA report found that despite heavier investment, enterprises experience lower success rates compared to mid-market companies.“By contrast, mid-market companies moved faster and more decisively. Top performers reported average timelines of 90 days from pilot to full implementation. Enterprises, by comparison, took nine months or longer,” the report found.

As for larger organizations, agentic AI could represent the best path forward for your AI initiatives. The McKinsey report thinks agents will be the transformational technology, and organizations that figure out how to build and manage agents most effectively, while navigating the organizational change management process, will be most successful, something that’s a lot easier said than done.

“AI agents mark a major evolution in enterprise AI—extending GenAI from reactive content generation to autonomous, goal-driven execution,” the report found. The companies that are succeeding with agents so far, have rearchitected workflows to take maximum advantage of the new technology.

“Realizing AI’s full potential in the vertical realm requires more than simply inserting agents into legacy workflows. It instead calls for a shift in design mindset—from automating tasks within an existing process to reinventing the entire process with human and agentic coworkers.” (If you want to see some real world success stories, I encourage you to look at the full report.)

New ways of thinking

Arvind Narayanan,  a professor of computer science at Princeton University and co-author of the book, AI Snake Oil (which I highly recommend), joined Tim O’Reilly recently for a podcast. He said the speed at which we adopt AI depends on many factors including “the rate at which human behavior can change, and organizations can figure out new business models.” This double whammy is what tends to hold most companies back. 

Narayanan says we need to stop concentrating on AI companies. “There’s too much of a focus on the AI companies in thinking about the future of AI. I’m talking about all the other companies who are going to be deploying AI,” he told O’Reilly.

He outlined four stages of adoption: In stage one, we improve the model's capabilities (check). In stage two, which we are currently in the early days, we begin to build products that take advantage of those capabilities. As he says, we are still searching for the right abstractions and the right interface for AI-fueled products. Whoever figures this out is going to have a tremendous advantage.

In stage three, early adopters begin to figure out what works and what doesn’t, and finally in stage four, for widespread adoption to occur, there needs to be substantial personal, social and business adaptation to the technology. “Individual users need to adapt; industries as a whole need to adapt. In some cases, laws need to adapt,” Narayanan said. All of this will not happen overnight.

The companies doing this well are rethinking everything. It makes sense that smaller companies are the most successful because they are for the most part unencumbered by legacy technology (a subject I plan to address next week). Startups in particular can build an AI-first company from the start, but most companies don’t have that luxury. How widespread AI adoption becomes will depend on how well companies and their employees adapt to the changes the technology will inevitably bring.

~Ron

Featured photo by Jungwoo Hong on Unsplash