How I stop an LLM from inventing cloud services
Ask a language model for an AWS architecture and you get a confident answer. Sometimes it includes a service that does not exist, a feature that is not real, or a database floating outside any VPC. A diagram makes this worse, because a clean picture looks more trustworthy than the text it came from. I built CloudArchitect Lite to turn a plain-English description into an AWS diagram. This post is…
A language model user built an app called "CloudArchitect Lite" to generate AWS diagrams based on plain-English descriptions. The app uses a curated catalog of 38 AWS services, each represented by an id, icon, and relevant configuration keys. The model cannot create non-existent services, as it generates against a structured-output schema built from the catalog.
If the user requests a service not in the catalog, the app labels it as an unresolved object, clearly marking it as non-existent. Before release, the app underwent rigorous testing with 22 scenarios, checking for required services and any errors in rule findings. The Claude Haiku model showed the best performance in terms of passing automated checks (100%) and receiving high scores in blind reviews (median 2.00).
Claude Sonnet also performed well but had a slightly lower median score (2.57). Although Haiku had a few missed services, these were the only rule violations. The reviewer panels consisted of Claude models reviewing Claude output, which may have introduced some bias. Users can run the app themselves using Docker and their own Anthropic API key, or try it in a browser at the provided link. The source code is available on GitHub.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
