So Agentic Systems Are Messing Up Your SLO Framework

Agentic systems, which are powered by artificial intelligence, are causing issues in service level objective (SLO) frameworks used by engineering teams. The systems rely on the notion that identical inputs always result in the same outputs, but agentic systems can produce different, plausible outputs. This can lead to quiet failures where the system appears healthy on dashboards, but an agent has quietly produced outputs that fall outside acceptable ranges of correctness.

Traditional SLOs are built on deterministic contracts that are not suitable for agentic systems, as identical inputs can produce different, correct outputs or cause quiet failures. Traditional SLOs are dependent on deterministic contracts, but agentic systems violate this assumption because identical inputs can produce different, plausibly correct outputs or cause quiet failures that never appear as errors.

While infrastructure metrics still matter, they cannot tell the whole story when part of the system is probabilistic by design. A second layer of measurement for behavioral correctness needs to be added on top of existing infrastructure metrics. This requires platform teams to start measuring reliability at the infrastructure layer, not just at the model layer, in order to catch nondeterminism problems that exist simultaneously across layers.