Table of Contents
The failure rates are staggering
Recently, an MIT report, The GenAI Divide: State of AI in Business 2025, grabbed attention. It found that only 5% of enterprise-grade generative AI systems reach production—meaning 95% fail during evaluation. Another report from Gartner suggested that 40% of agentic AI projects will be scrapped by 2027.
And in simulated office environments, LLM-driven AI agents get multi-step tasks wrong nearly 70% of the time.
Agentic AI—long thought capable of automating most contact center workflows—is starting to look like it could turn into a bad science experiment. And these won’t be the last reports of AI agent failure in contact centers.
The causes aren’t random
But why? There are clear indications these failures are not random. Both MIT and Gartner point to similar root causes: projects driven by hype, poor alignment with real workflows, and weak governance or risk controls. MIT calls it the “learning gap”: enterprises don’t know how to design AI systems that actually learn and adapt, so most pilots stall out. And Gartner highlights rising costs, unclear value, and a market flooded with “AI agent–washed” products. The reality is that enterprise adoption demands more than LLM-driven agents; it requires systems built to integrate, adapt, and improve over time.
Why listen to a marketer?
So you might ask: Why keep reading a blog post written by a marketing professional, especially one who represents an AI for CX company?
Because it’s my job to read the market. And that means keeping up with the marketing trends. Here’s what I see. The hype and the noise in the AI market make it much harder for truly innovative solutions to be heard. I see some vendors exploiting the confusion by “AI agent washing” their solutions. I hear about AI agent systems that can’t do basic knowledge retrieval or connect to existing CCaaS reliably, yet still end up on many buyers' short lists simply because they’ve made grand promises on paper.
That makes it harder for you to find the right solution—and harder for those of us with successful enterprise deployments, who have taken the time, effort, and expense to refine the process and improve real-world results, to get our message across.
In this blog post, I want to share some of my own observations about the market and what you should know about AI solutions and platforms for CX.
An example of AI agent washing
Case in point: I recently came across an ad on Instagram promising a CX AI agent with on-brand responses and “no hallucinations.”
The process seemed simple: import your website content, test how the AI agent responds in a chat window, and then, supposedly, activate it. Voilà—problem solved!
Except that’s not how it works. Far from it.
Here’s where the marketing confusion comes in. Most customer-facing AI agents are generative, because that’s how they create a conversational experience without relying on predetermined flows. But, that’s also why they inherently would hallucinate. The risks, though, can be dramatically reduced, but never reduced to zero. If you are interested, this blog post written by my colleague Heather is a great read to get a better understanding of hallucination and strategies to properly manage it.
And you can’t simply ask a generative AI agent a few test questions, get a correct answer, and declare the agent ready to go live with your customers. Safety requires much more robust testing, including simulated real-world scenarios, along with continuous monitoring and safeguards. A simplistic one-time check won’t cut it.
I dug into the website for this particular solution, and indeed, nowhere did they claim to use generative AI. So, of course, it doesn’t hallucinate. That likely means this so-called AI agent relies on natural language processing (NLP) to determine customer intent, and then follows deterministic flows. It completely lacks the power of a true generative AI agent.
And that takes us to a word that vendors can take advantage of, especially in CX: agent. In my mind, the name “AI agent” implies the solution is “agentic” and therefore, capable of taking actions autonomously without or with minimal human guidance. With this particular so-called AI agent, there is zero indication that it’s capable of any autonomous action. The word agent is used simply for its current marketing power, positioning this CX bot alongside more capable generative AI agents for customer service.
So, at the end of the day, this “AI agent” is really just an “AI agent–washed,” glorified chatbot—neither generative nor agentic.
Navigating the CX AI agent marketplace
As fast as this confusing AI agent market is moving, choosing not to deploy an AI agent is not ideal. More organizations are taking advantage of AI agents every day, and the 5% who get the deployment right will gain a tremendous advantage over their competitors.
So, here are a few things you can do to move forward safely and protect yourself against the marketing tactics I’ve mentioned, directly from a marketing professional.
1. Question the terminology
When vendors tout their solutions as being an AI agent, investigate what they mean. Does the AI agent leverage generative AI? Or just some kind of AI (e.g., NLP)? Is it truly agentic, or does the vendor focus primarily on its conversational nature? If it is agentic, does it simply retrieve information to serve up to customers, or is it connected to other systems through APIs so it can act to solve customer issues?
For AI agent platforms, if the vendor says they keep a human in the loop, does that mean human agents are still just an escalation point for an AI agent that’s incapable of acting autonomously? Or does it mean there is a structure in place for human agents to step in and guide the AI agent, without taking over the interaction completely—so the AI agent is still largely autonomous?
We put together a short guide that we’re still expanding to better explain some of these terms that you definitely want to keep in mind when you’re evaluating vendors.
2. Look for proof
There are a few places where you can find proof that a company has a strong vision and the technological maturity to deliver innovative solutions. But, keep in mind that the AI agent platform market is still young, which means you’ll need to look beyond the most obvious sources to spot reliable vendors.
Analyst reports
Look for companies that are cited in analyst reports for their direction and vision, as well as their technology and ability to execute. This indicates that the company actually thought through the deployment and understands the impact of the deployment.
Analyst reports with vendor evaluations are also a reliable source of information, but keep in mind that the research cycle for those can typically be long, and some innovative players will be noticeably missing from the reports that are coming out just now.
Enterprise customers
Enterprise logos are another good piece of evidence the vendor is trusted, but also keep in mind that large enterprises are sometimes reluctant to reveal their use of AI technologies, given that this is still a new market. So, look beyond the logos on the vendor’s home page to see if the vendor has customers who appear at events or presentations with them. The willingness to share a stage with the vendor is a kind of endorsement of their product.
Case studies
See if vendors have case studies that cite concrete outcomes, specifically ones reported with mission-critical CX metrics, and not just fluff pieces that talk broadly about the vendor’s products. Similarly, look beyond names and logos. And focus on whether the case study is based on a product that’s live in production, and not just an internal deployment in a safe environment without the stress test of the real world. You’ll need to read between the lines.
3. Meet with vendors as early as possible
Once you put together a shortlist of AI agent vendors, meet with the vendors as early as possible. There are three big reasons not to delay.
One, a conversation can easily reveal whether the solution is just a prototype or a mature platform working in production. You will also have better insights on reference customers beyond what’s visible publicly, and understand their deployment process.
Two, you can connect your stakeholders with the vendor early. Selecting an AI solution usually involves multiple stakeholders; a vendor with experience can help facilitate your internal conversation and answer questions other stakeholders have early on. It will also help you gather internal momentum and support.
Three, some clear red flags are much easier to spot after a chat with a vendor. For example, stay away from vendors who:
- Can’t provide reference clients or production use cases.
- Focus only on headcounts, and promise to replace all your human agents.
- Push you to hand over all customer interactions, instead of focusing on the use cases where AI agents will excel.
- Can’t articulate how their solution delivers value and concrete CX outcomes, beyond “sounding human.”
- Do not offer any POV, free or paid.
4. Make sure the vendor offers POV
Doing a Proof of Value (POV) is the only way to validate the technology and to find out if the AI agent works as intended (vs. one that just looks good on paper). It gives you insights into whether the AI agent can successfully integrate with your existing tech stack and what resources are required to deploy a use case. Finally, you will also get a chance to work with the vendor team and better understand whether they will just leave you with a broken toy, or provide expertise and knowledge to carry you through the process.
It is incredibly important to do a POV, especially given AI agent platforms are relatively new to the market. If a vendor doesn't offer one, then you have to take on the risk of committing to the vendor before knowing whether their technology is mature enough to safely and reliably handle real customer interactions. And given AI investments typically require stakeholders across departments, a failed project can cause real damage to internal support for AI systems and make it even more difficult to rally for future projects.
Some vendors offer a free POV, while others offer paid ones. It is important to consider the following pros and cons.
Free POV:
- Low barrier to entry, fast experiments
- Usually limited in scope, or with a use case that doesn't fit what you really need
- Could have lower vendor commitment, as they are not getting paid
Paid POV:
- Requires budget allocation, more internal approval
- Usually clear delivery timeline, milestones, and resource commitment, as the vendor is paid and motivated to deliver
- Usually a more realistic environment with deeper integration, which is necessary with agentic AI evaluation in CX
Finding the path forward to 5%
While the MIT and the Gartner reports highlight today’s failure rates, there is a clear path to the 5% with successful pilots leading into production. I expect this number to grow soon as companies have a better understanding of how to identify and deploy AI solutions in an enterprise environment, and those who do this well, early, will have a full advantage over their competitors. By demanding clarity on terminology, validating vendor claims with proof, meeting vendors early, and insisting on a POV, you can move forward with an AI solution that truly addresses your contact center challenges—and delivers both CX outcomes and business value.
