Building an AI Customer Support Agent with Memory

Customer-support systems can rapidly provide answers, but a truly effective support agent requires more than just producing timely responses. It must comprehend the context surrounding the issue. While developing ResolveIQ Lite, my focus was on enhancing the AI's ability to use relevant customer history without introducing unverified information.

Consider this scenario: a customer once stated, "My order A104 arrived damaged. I need a replacement." However, later they wrote, "I still haven't received my replacement." Without memory, the AI would be uncertain which order the customer was referring to. With relevant memory, it could connect the message to order A104. Nevertheless, there's an important constraint: the AI must not fabricate details.

For instance, it shouldn't claim, "Your replacement was shipped yesterday and will arrive tomorrow," unless that information is present in the available history. This became the primary objective of my implementation.

The system functions as follows: A colleague developed the memory/retrieval aspect, while I concentrated on the agent.py component. This module takes the recalled memories and the current customer message and generates the response. The retrieved information is incorporated into the LLM prompt as PAST HISTORY. This separation allows the model to clearly distinguish between current data and previously remembered information.

Prompt engineering proved crucial. Simply providing an LLM with memory does not ensure it will use it appropriately. I introduced guidelines instructing the agent to: Utilize previous history when relevant. Reference earlier orders or problems as appropriate. Never fabricate details such as tracking numbers, dates, or policies. Never claim to remember something that is unavailable.

Never invent a completed refund or replacement without evidence. Request missing information when necessary. Maintain concise, courteous, and precise responses.

A useful test involved comparing the same message with and without memory enabled. With memory, the agent could accurately identify the relevant earlier incident. Without memory, the agent should request the missing order details instead of assuming it recalls A104. This confirmed that the memory was indeed influencing the response.

However, a failure during testing taught me a valuable lesson. I encountered a response like, "Thank you for confirming it's order A104." The customer had not confirmed order A104. The model had correctly retrieved A104 from memory but treated the retrieved information as if the customer had just provided it. This highlighted that memory is not just about retrieval.

The AI also needs clear instructions about what is known, what is remembered, and what has been confirmed by the customer. I refined the prompt accordingly and continued testing. Additionally, I implemented a fallback mechanism so the response generator does not rely solely on one model. The agent attempts the configured models in sequence: if one model fails, the next can be tried.

All returned responses undergo cleaning before being presented to the customer. Through this process, I learned that memory requires clear boundaries. Simply increasing the context provided to an LLM does not guarantee improved accuracy. Testing with and without memory is essential to determine if retrieved history actually impacts the response.

Testing failure cases, such as queries about missing tracking numbers, delivery dates, or policies, can reveal when the model starts making up information. Prompt engineering is a fundamental aspect of system design, defining what the agent can and cannot claim. Finally, testing the fallback mechanism is crucial, as it is only beneficial when the configured models are actually accessible.

The LLM component of ResolveIQ Lite now operates on a simple principle: the goal is not merely to make the AI sound confident, but to ensure it uses customer history when available, requests information when it is not, and avoids presenting assumptions as facts. This was my primary lesson from building the LLM layer of ResolveIQ Lite.