You Can't Test Money Controls With a Free Model
Every budget gate in my v0.2.0 field test was green. Every spend-velocity guard passed. Every tenant cap held. Every ROI flag was clean. All of it was green-on-vacuum. The local model was priced at $0, so no run could ever breach. My "budget enforcement" was a unit test about arithmetic — it proved the math, not the gate. When I sat down to test the cost system on HivePlane v0.2.0 — four budgets,…
The article reveals a hidden flaw in budget gate testing when using a free model. Despite all checks and controls appearing to pass in a simulated environment, the reality changes once the model costs money. Four budgets, spend-velocity guards, tenant spend caps, cost-per-completed-task metrics, ROI flags, and an attestation-based result cache are all rendered untestable when the underlying model has zero cost.
The green test proving the math is ultimately meaningless if the model's cost is never enforced. A zero-cost default model makes each gate structurally untestable because there's no spend to exceed, no velocity to detect, no caps to breach, no ROI to flag, and no cost to flag. The test only demonstrates arithmetic correctness, not actual enforcement.
The fix isn't a code change, but a test-design feature: an opt-in "priced profile" that prices the local model, allowing the budget gate to operate at the right stage, before expensive work begins. Ignoring this can lead to real-world problems where budgets are never refused, spend caps are unenforced, and ROI metrics are meaningless. The author emphasizes the importance of pricing the model in test profiles and binding caches to attestation validation to prevent silent regressions and ensure actual budget enforcement.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
