Pilot fatigue, aimless experimentation, and failure rates have many organizations shifting generative AI investments toward more targeted — and promising — business use cases.
Generative AI playtime may be over, as organizations cut down on experimentation and pivot toward achieving business value, with a focus on fewer, more targeted use cases.
Nearly nine out of 10 senior decision-makers said they have gen AI pilot fatigue and are shifting their investments to projects that will improve business performance, according to a .
Organizations will still experiment with new gen AI pilots, but a more targeted approach focused on use cases specific to their businesses is increasingly becoming central to IT leaders’ AI strategies, says Andrew Wells, chief data and AI officer for NTT DATA North America.
In some cases, pilot failure rates of 50% or more have forced organizations to rethink the number of pilots they spin up, Wells says. In an April survey, IDC found that, on average, organizations had launched 37 AI proof-of-concept projects, with a small minority reaching production.
“Either you didn’t have the right data to be able to do it, the technology wasn’t there yet, or the models just weren’t there,” Wells says of the rash of early pilot failures.
In other cases, the pilot wasn’t commercially viable, he says. “You would build the POC, but the efficacy of the solution didn’t necessarily pan out with the original hypothesis,” he adds.
Moreover, more than a third of IT pros have said practical value hasn’t been the aim of AI projects they’ve worked on — but to show investors and stakeholders their organization is doing something with AI.
And the glut of gen AI pilots going nowhere is proving a drain on resources, Wells says.
“When we go into most companies, their backlog of gen AI use cases [is substantial], specifically in the hundreds,” he says. “They’re being more purposeful about what they want to spend the time and energy and dollars on versus, ‘Let’s just experiment and see what the technology might be able to do.’”
Underestimating the cost
Too often, organizations have launched AI pilots without thinking about the hidden costs, says Courtney Schuyler, CEO and co-founder of SkyPhi Studios, a digital transformation advisory firm.
Launching several pilots in a short time not only can cost a lot of money but also often leads to a loss of employee productivity, as they struggle to learn how to use the new technology.
“Oftentimes, what I see is organizations jump on this technology hype before really slowing down and thinking strategically about how this can be utilized within their organization, what strategic path they should be taking, as well as just looking at the overall change fatigue the organization might be going through based on other technological implementation,” she says.
Getting a project to the implementation stage is only part of the battle, Schuyler says. Getting the necessary employees to adopt a new gen AI tool is a huge step.
“What’s important is to really realize your investment and the costs associated with it, everything from the cost of the tech to the cost of helping people to adopt it,” she adds. “Taking the time to look at that budget for it and plan for it, from my perspective, is more important than just jumping right in and potentially losing millions of dollars, because it’s just not as effective as you’d hoped it to be.”
Spending still increasing
Even with mixed results in the past year, many companies are planning to increase their gen AI spending in 2025 and beyond. In the NTT DATA study, 39% of those surveyed say they now have significant investments in gen AI, with the percentage rising to 61% within the next two years.
The NTT survey aligns with a , which found that 62% of companies are planning to increase their AI budgets in 2025. Still, despite pilot fatigue more than a quarter of those surveyed say their companies plan to launch more than 20 AI pilots in 2025.
Although experimentation will continue, many organizations are likely to focus on projects that give them a competitive advantage, not general HR, digital assistant, or chatbot projects, says Dev Nag, CEO of QueryPal, a support automation company.
As early AI projects, many IT organizations tried to and HR AIs, Nag says, but some are now offloading those functions to AI vendors.
“We saw that in a microcosm, people trying to build [chatbots] themselves with teams that really weren’t dedicated to support and didn’t have any kind of AI background,” he says. “There was this huge over-investment. We turned corporations almost into VCs,” funding IT projects as if they were startups.
But having every organization build their own HR AI doesn’t make sense, Nag says.
“Would you really rather have10,000 enterprises go off and try to build a customer support agent and an HR agent, and a finance agent?” he says.
Instead, many organizations seem to be moving toward a smaller number of gen AI pilots that focus on their unique needs, he adds, rather than commodity chatbots.
“For most companies, if it’s not super relevant to your bottom line, it’s going to be a distraction and be a failure,” he says. “You’re going to lose the people, because they’re going to be thrown into this thing where it’s like, ‘We expected it to be a center of success.’”
Specific needs
Aaron Schroeder, director of analytics and insights at contact center IT vendor TTEC Digital, sees some of the same trends. Much of the major publicized advancements in gen AI are coming from general-use models focused on individual use cases, not complex business uses, he says.
“These models and features are grounded in broad knowledge from across the internet, rather than in specific domains and contexts,” Schroeder adds. “This causes many leaders to see how emerging AI solutions help them in their daily life, but there’s still a gap between that, and seeing AI solutions be meaningfully productive in a hyper-specific, industry-oriented use case that requires knowledge of how your company operates.”
This gap is driving business to move pilots away from general-use projects to ones more aligned with areas that drive value, he says.
“The most successful approach we’ve experienced is in designing the governance for AI pilots and solutions up front, at a higher level — whether that’s to accelerate productivity, drive cost savings, increase revenue, or drive better customer experiences,” Schroeder says. “By identifying the key tenets up front, it becomes easier to maintain alignment on multiple projects that are occurring simultaneously in a company.”
