Enterprises continue to struggle to find AI success

Despite the hype around generative AI, studies continue to suggest the majority of large organizations are struggling to find ROI and project success. The question is why. Two recent studies from IBM and AWS suggest there are a number of problems. While these studies highlight a range of challenges, in my view, the core issues boil down to data quality, a huge skills gap and lack of commitment to change management.

In a press briefing this week, IBM CEO Arvind Krishna pointed to a CEO study the company commissioned, conducted over the last several months, which found that just 25% of AI projects have delivered the expected ROI. It was in line with studies we’ve seen going back to the end of last year. These studies have consistently found a success rate of around 25%, whether it relates to ROI or moving a product beyond the proof of concept or experimentation stage, where so many companies seem to be stuck, and into production across the company.

In the study conducted by AWS, the cloud infrastructure vendor found similar results. As you can see from the chart below, 23% of AI projects graduated into company-wide usage, what they call "workflow integration."

IBM's Krishna believes that more quantifiable success is coming, but so far the data doesn’t necessarily bear that out. “So the change over the last months is that people are stopping experimentation and focusing very much on where the value to the business is right now,” Krishna said at the IBM press event this week.

AI is grabbing budget

The problem certainly isn’t access to budget. According to the AWS report, companies are devoting more budget to AI than security and it’s not even close at 45% to 30%. While IBM didn’t ask respondents to break it down by category, it did find that respondents expected their AI budgets to grow twice as fast as last year. Given reports of hefty AI budgets, that would suggest quite a bit of money being devoted to AI.

It’s worth noting that it’s tough to separate out which parts of the budget are devoted purely to AI, which could include security, storage, compute and other technologies as part of the overall cost of implementation, something that Rahul Pathak, VP of data and AI at AWS acknowledged.

“On some level it makes sense that generative AI is essentially top of mind, and is one of the key criteria for investing and a key budget priority,” Pathak told FastForward. For him, that didn’t necessarily mean there was deemphasis on security or any other technology line item, but he believes it's more about prioritizing AI applications as a top budget item.

The persistent data problem

If it’s not money, perhaps it’s data quality. One of the biggest challenges holding back companies involves getting access to good data to train large language models, something we have discussed quite often at FastForward. It remains an issue for a lot of companies, who are struggling to get their data in order before they can take advantage of generative AI and large language models.

“Many organizations lack the foundational data environment needed to support transformational and powerful technologies like AI,” according to the IBM report.

AWS’s Pathak also encounters data challenges in his conversations with customers.
“One of the things I always talk to customers about is that data is your key differentiator when it comes to AI applications because anyone with a credit card can interact with an AI model. It's really about what unique data you can bring to bear,” he said.

Bridging the AI skills gap

One other major problem is an AI skills gap, whether we are talking about engineering or any other role in an organization. While companies increasingly expect their employees to be using AI tools in their work, it’s not a simple matter figuring out which tools are best for a given task or to become adept at using them, even for technically proficient workers. Yet AWS found that 92% of respondents expect some level of AI skills proficiency, however you define that, in job openings for the coming year.

Pathak says that leaders need to help employees start thinking about how to develop AI skills. “I think being intentional about helping people develop the skills they need to get the breakthroughs that we know are possible is super important,” he said. 

The IBM report found that 54% of CEOs were hiring for AI-related jobs that didn’t even exist a year ago (and some newer jobs like prompt engineering have come and gone already, per the WSJ). The AWS report found that over 50% of organizations surveyed had training plans to get their employees up to speed, but more surprising were the organizations without any such plans.

The change management challenge

Perhaps one of the biggest problems identified in the AWS report was a dearth of change management plans with only 14% of companies saying they had a plan in place today. When you consider this could be a far bigger transformation than any attempt at digital transformation over the last decade, lack of a clear plan could end up holding enterprises back.

While the survey found that 54% expect to have a plan in place by the end of 2025 with that number increasing to 76% by the end of 2026, it’s still surprising how few organizations are putting these plans in place ahead of AI mandates. Perhaps it’s not a coincidence that Pathak sees a direct correlation between companies with good change management plans and AI success.

“I do think customers need to be thinking carefully about change management, and the most successful customers we've seen getting AI broadly adopted internally or into production at scale, have been thinking about this,” he said.

The IBM report backs this up. “But, no matter the pace of change and depth of disruption, leaders still need to push their organizations forward—finding new avenues to efficiency and more direct paths to growth. In this complex environment, it’s not always clear how CEOs can keep their organizations standing strong and focused on the future.” Such a situation begs for change management on the part of executives.

None of these are minor issues. While IBM’s CEO may declare the age of AI experimentation over, words alone won’t make it so. True progress will require a full-scale commitment to improving data quality, closing the skills gap and tackling deep-seated change management challenges. Until that happens, AI projects could continue to languish inside large organizations.

Featured image by Rahadiansyah on Unsplash