Statewave Guide — Day 4: We passed the safety test. Then failed the product test
Built 27 Aug 2026 · written up 14 Oct 2026 Yesterday I asked: Can technically safe product knowledge survive contact with an actual user? Not particularly. 😅 One model, one fixture, same as yesterday: every number below comes from anthropic-opus-5 against a single application we wrote ourselves, with patterns we knew to look for. Wrong-feature claims accepted 0 Unsupported claims accepted 0…
On the fourth day, the Statewave Guide had passed a safety test, but failed a product test. The test revealed that while the product was technically safe, it was not useful or easy for humans to understand. The model scored 30/30 for verified claims, but only 1.43 features were found to have a verified capability. The remaining features were considered mediocre, with 14 outputs being too technical and 10 explicitly telling users that no capability was verified for the feature.
Despite the technical grounding, the model's language was deemed linguistically deceased, making it difficult for users to understand. The architecture was updated to include a Product Model, Guidance Compiler, and Renderer to address this issue. The key takeaway from this day was that technically correct data structures do not automatically know how to communicate effectively with humans.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
