Stage 5 of enterprise AI adoption: governance hardening

This is stage 5 of AI adoption, known as governance hardening. It focuses on four key components: policy and gates, audit ledger, halt and budget caps, and cost and quality metrics. Governance begins here, with logging and rate limit & budget already in place since stage 1, continuously recording calls and charging against a budget.

The stage introduces comparison, attribution to a rule version, and the ability to stop a running process. The company in the example is Qingchuan, a team of 40 people, including 8 engineers and 2 marketers. Qingchuan uses the gateway for logging in, carrying roles, scoped retrieval, sandboxed agents, and approval records for writes and sends.

The first box, policy and gates, deals with rule versioning, comparison between declared rules and actual implementation, and drift detection. The drift can be invisible from within the rule. Version control, review process, and scheduled comparison are essential for effective governance. The policy engine uses explicit unknown and conflict states instead of folding them into allow or deny.

The second box, audit ledger, adds the fifth field to track which rule allowed a specific action. An append-only ledger, written by the acting system, ensures accountability and traceability. The ledger is read-only, safeguarded against unauthorized access.

The third box, halt and budget caps, distinguishes itself from a simple rate limit by stopping a run already in progress, rather than simply refusing the next call. The stop is at the run level, affecting multiple actions if necessary.

Lastly, the cost and quality metrics box measures the performance of the governance system, focusing on p95 (95th percentile) rather than averages. This ensures that the system accurately evaluates and optimizes the system's efficiency and effectiveness.