Presentation: From AI Agent Demo to Production: Automated Testing and Evaluation
Zhou Yu discusses why AI agents stall in demo phase and shares how simulation-driven testing solves compliance and reliability bottlenecks. Learn how Columbia and Arklex AI use synthetic user personas, trajectory entropy, and automated CI/CD pipelines to evaluate multi-turn agents, catch edge cases before deployment, and scale self-learning workflows in production. By Zhou Yu
InfoQ's event titled "From AI Agent Demo to Production: Automated Testing and Evaluation" featured Zhou Yu, a professor at Columbia University and founder of Arklex AI. She discussed the challenges of AI agents remaining in demo phase, testing and evaluation methods, and real-world applications. Yu highlighted simulation-driven testing as a solution to address compliance and reliability issues in multi-turn agents like conversational agents such as Walmart's Sparky and Amazon's Rufus.
She demonstrated how these agents can leverage tools, contextual understanding, and user data to provide personalized experiences, similar to human salespeople in a physical store. Additionally, she showcased voice agents, like those assisting customers with credit card applications based on their shopping habits and preferences.
Written by urgent.news from InfoQ's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
