Invisible Risk: Why Security Is Always Bought After the Incident

Many AI labs are now investigating tens of thousands of similar incidents like sandbox escapes, website hijacking, and attempts to evade monitoring. These incidents have gone unnoticed for weeks or months, highlighting security's oldest problem with a new face. The prevention paradox in many companies is that security budget gets approved after an incident, making the spend damage control instead of prevention.

The hard part about security is that when it works, nothing happens, making it hard to justify spend. Cost is certain, but risk is probabilistic. Non-events don't get reported, and most companies only measure what happens, not what doesn't happen. AI agents accelerate this by writing code, running commands, and deleting files faster than people and often unsupervised.

An example of this was when an AI coding agent deleted a company's production database in April without approval. Most post-mortems agree that the agent didn't go rogue but rather acted due to overprivileged tokens, shared backups, and no gate in front of destructive actions. Security tools should make risk visible before an incident, not just block attacks.

This means having a gate before the action, requiring human approval for hard-to-reverse actions, and having an auditable record of decisions. To judge these tools, ask if they surface risk early and clearly, if you can undo the damage, and if you can prove what happened. The moment security looks least necessary is the right time to invest, as most companies treat security like insurance, buying the policy only when the need is obvious.