Your company’s AI needs a scoreboard
I’m talking to many companies these days that are trying to incorporate AI into their business practices. Most of them are still in what I call “the administrative phase,” when they just try modest automations that allow some workers to draft documents or presentations, answer emails, or do research on certain topics, but given my recent articles arguing for companies to go much farther, I’m…
Many companies are now looking to integrate AI into their operations, moving beyond mere automation to a more strategic approach. The primary concern for these businesses is whether the AI will be intelligent enough, but a more useful question may be: How does the AI determine if its actions improved the business? An example of this is optimizing a salesperson for revenue, margin, or retention.
While improving one metric can often harm another, businesses require outcomes-focused evaluation rather than just output-based measurement. Metrics such as customer churn, margin, conversion, claims accuracy, delivery times, or customer lifetime value are more relevant to business success than output measures like answer quality, task completion, latency, or cost.
Companies need to establish a clear scoreboard to measure the impact of AI on their business, as poorly chosen metrics can become dangerous when aggressively optimized. Continuous evaluation becomes essential, with AI agents needing to measure outcomes and improve based on those results, much like how management operates through objectives and feedback.
Companies must also consider the potential trade-offs between different objectives, such as margin, churn, returns, compliance, or brand image. Ultimately, the success of AI implementation depends on clear objectives, continuous measurement, and a clear understanding of the consequences of the AI's actions.
Written by urgent.news from Fast Company's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
