Table of Contents
This is part of a blog post series focusing on the AI usage gaps identified within the MIT NANDA report, The GenAI Divide: State of AI in Business 2025.
A recent MIT NANDA report, The GenAI Divide, noted an interesting gap in AI usage. Among the companies surveyed, 90% of employees are using consumer AI tools in their work. But only 40% of companies have purchased an official LLM subscription.
That discrepancy isn’t a simple case of cost-conscious budgeting. It all comes down to how the employees are using the AI—and where they run into its limitations. According to the report, employees will use an AI tool for brainstorming and other low-stakes tasks, but abandon it for “mission-critical work due to its lack of memory.”
Without memory, AI tools aren’t much help with complex enterprise workflows. In the contact center, that means they start every interaction from scratch. They can’t tailor their responses based on all of the relevant context. And they can’t learn, adapt, or evolve.
Understanding AI memory
Memory is a critical part of what allows an AI agent to retain context, recognize patterns, and adapt over time. The larger and more relevant the memory is, the better an AI agent will become at personalizing service and evolving as conditions change.
On their own, large language models (LLMs) can’t retain new information. So, memory must be added through other components in the agentic platform. It’s helpful to think about this memory in two broad categories: short-term memory and long-term memory.
Let’s take a closer look at these categories of memory in the context of agentic AI for customer service.
Short-term memory
An AI agent’s short-term memory temporarily holds information during a customer conversation. It can include data retrieved from other systems, like the customer’s account details, as well as relevant information from what the customer has said. The ability to hang onto this information provides consistent context for multi-turn conversations. That leads to better customer service and a higher resolution rate.
The amount of information an AI agent can temporarily hold is called the context window. It’s like a rolling buffer that holds a limited amount of recent data. As the conversation progresses, new data is added, and some data drops out. The size of the context window and how it’s optimized determine how long the conversation can continue before the AI agent loses information from earlier in the exchange. Hold too little information at a time, and the agent lacks what it needs to resolve the customer’s issue and must ask for the information again. But overfill the context window, or fill it with the wrong information, and that can degrade the AI’s decision-making.
Engineering and optimizing the context window is a hot topic in AI agent development right now because it has an enormous impact on the agent’s performance.
Long-term memory
When a customer conversation ends, the information in the context window does not persist. But some data from each interaction would be useful to have long-term. Storing data for future reference is the domain of long-term memory, which allows an AI agent to store and retrieve information from previous interactions and use it to provide more personalized service to the customer it’s currently interacting with.
In addition to general information like definitions and rules, long-term memory can store task-related procedures that reduce computation time and allow the agent to respond faster to specific inputs. Long-term memory can also log key elements of the customer’s behavior, the agent’s actions, and the interaction’s outcomes in a structured format that the AI agent can access during future customer conversations.
For example, the long-term memory for an AI agent at an airline might include nuanced information about a frequent flyer’s seat preferences—window when traveling alone, but aisle when traveling with her son. After retrieving that information, the AI agent could ask proactively about booking the aisle and center seats for this family trip.
In this way, long-term memory will enable a new level of personalization and allow the AI agent to adapt and evolve over time. In practical terms, this kind of long-term memory is in the early stages of development for contact center AI agents. But more sophisticated memory architecture is coming in the near future.
Why AI memory matters
Memory isn’t just about storing information. It’s about creating systems that can retrieve information from earlier in the conversation or even from past interactions, determine its relevance to the current moment, and use it as context to adapt and provide better service. Optimization of both short-term and long-term memory will make the next generation of AI agents far more effective and useful. It will enable more meaningful and satisfying experiences for customers, and expand the AI agents’ ability to work autonomously and adapt intelligently to improve performance over time.
That’s why memory will become a key differentiator that sets top-tier AI agents apart.
How AI memory affects customer service
The quality of an AI agent’s memory management can either elevate your automated service, or leave customers disappointed. As vendors continue to optimize their agents’ memory, here are some of the benefits you can expect.
Improved efficiency and satisfaction
The more an AI agent’s short-term memory is optimized, the more efficient and reliable it becomes. It won’t need to ask the customer to repeat information, even late in a very long, complex conversation. It holds enough information to keep the interaction flowing toward resolution. That reduces customer effort and increases satisfaction.
More personalized customer service
The AI agent will use data from past interactions, transactions, and other customer behavior alongside account details and preferences to customize responses and recommendations. In effect, the information pulled from long-term memory will serve as additional data points for the AI to predict customer behavior and more precisely adapt to each unique customer.
Continuity throughout the customer journey
Through strategic storage and retrieval of data from past interactions, an AI agent will build on previous conversations rather than starting each one from scratch. That saves customers time and effort and makes them feel known and understood. Rather than disconnected interactions, the experience will feel more like a persistent relationship, which will deepen the customer’s loyalty to your brand.
Increased trust and engagement
Personalized service makes customers feel like your brand cares. That instills confidence and builds trust. And it makes customers more likely to engage and share relevant information in their journey with your brand. That information will fuel even deeper personalization.
What customer service leaders should look for now
We’re still in the early days of focused research on memory management for AI customer service agents. So, many of the possibilities discussed above aren’t fully developed yet. But you should still probe for information about a vendor’s roadmap for developing their AI agent’s short-term and long-term memory. Specifically, ask about their plans and anticipated timelines for these capabilities.
- What’s your timeline for the agent to be able to retrieve relevant information about a customer from past interactions and use it as context in the current interaction?
- What kind of information are you planning to make available to the AI agent from previous interactions (customer’s tone, stated preferences, purchase history, issues, resolutions)?
- Will your AI agent be able to aggregate information across channels, including both digital and voice?
- How will your AI agent share aggregated information as context with human agents?
- How will you ensure transparency into what data is stored and how the AI uses it?
Memory as a key differentiator—and value driver
Improvements in LLMs have slowed, and that plateau will persist. But the capabilities of AI agents will continue to expand, in large part through advances in memory optimization. Top solutions are already beginning to support proactive problem resolution and hyper-personalization to deepen customer relationships, build loyalty, and fuel business growth.
AI agent platforms with a sophisticated memory architecture will soon capture a gold mine of critical structured data about your customers. That data will shift the role of the contact center from a mere support desk to a customer intelligence hub that delivers value throughout the enterprise.
The next wave of AI agents will use memory to plan complex workflows, anticipate customer needs, and capture knowledge that improves service and organizational agility. Enterprises that prioritize robust, adaptive, and privacy-conscious memory management in their AI agent investments today will gain an unprecedented edge. The time to embrace those priorities is now.
