Silent Failures, Not Crashes: What Building an Agentic System in Two Weeks Taught Us
I created this article for the purposes of entering the Google Cloud All Things Agentic Hackathon. Two weeks ago, at Elite Solution, we asked a simple question: why do AI support bots confidently answer with policies their companies never wrote? For a small business — a solar installer, a clinic, a workshop — half the real knowledge lives on the website and half in the owner's head. A bot trained…
In the Google Cloud All Things Agentic Hackathon, a small business owner from Moldova showcased a project called Foundry. The objective was to address the issue of AI support bots confidently providing information that a company had not explicitly approved. Foundry was designed to create a support knowledge base from a business website, detect gaps and contradictions, and then verify the information by asking a human.
Once the human approved the answer, it became active and the old entry was marked as superseded. The system refused to answer any question without verified human approval, ensuring reliable information was provided. The stack used for this project was built on Google Cloud in the europe-west1 region, utilizing various Google Cloud services such as Google ADK, Gemini 3.5 Flash via Vertex AI, Cloud Run, Firestore, Cloud Scheduler, and Pub/Sub.
The project also used Gemma 2 (2b-it) on a dedicated Vertex AI endpoint for question classification. One surprising realization during the development process was that the real risk in agentic systems is not crashes, but silent failures. Four times during the build, a feature appeared functional but was completely ineffective in practice.
These silent failures went unnoticed until a human interacted with the actual interface. This highlighted the importance of testing the actual user experience and not just relying on automated tests. As a result, before any feature is considered complete, the team now walks through the feature as a real user would. They also confirmed the root cause of any issues thoroughly before applying any fixes, as a blind fix could be riskier than addressing a known bug.
Additionally, the project incorporated adversarial personas and an evaluator to test the bot's responses. If the bot refused to answer a question correctly, it was considered correct behavior. This rigorous testing process helped ensure that Foundry would provide reliable information to business owners. Looking ahead, the team plans to integrate the Gemma classifier into the live pipeline, create a review page for owners to approve findings, and develop a continuous readiness score for the business owner to check before going live.
The project was built by Victor Axenciuc from Elite Solution SRL (Moldova) and demonstrated at the Google Cloud All Things Agentic Hackathon.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
