Urgent.News

What's breaking now, across hundreds of outlets.

AI

How AI Agents Shape Human Judgment Through Analytical Assumptions

An AI recommendation can be accurate and still steer your choices. How to question its priorities before approving decisions that affect your business.

How AI Agents Shape Human Judgment Through Analytical Assumptions

An AI agent once guided a founder in determining which customer accounts required more focus. The agent evaluated revenue, support expenses and renewal dates, then generated a sensible ranking. However, the agent unknowingly decided what constituted "deserving more attention." This could have been driven by prioritizing immediate revenue over customers with growth potential or interpreting support requests as costs instead of recognizing a need for assistance.

The founder verified the outcome, but the underlying objective received less attention. The true challenge lies in recognizing that a recommendation can contain unspoken choices that the recipient is unaware of making. For instance, when a company utilizes AI to pinpoint employees who might leave, numerous decisions are made before assessing the predictions' accuracy.

Those choices include whether every exit should be prevented, the status of employees with scarce skills or those whose development has stagnated, and the implications of employees moving to other organizations.

To address this, individuals should outline their desired outcomes and what they are unwilling to compromise. By making these choices explicit, one can question them. For the customer example, this means preserving long-term relationships while minimizing unnecessary support work. Once the objective is clear, the AI can assist in identifying reasonable alternatives.

For instance, if both approaches yield accurate forecasts, the conversation shifts to the company's commercial priorities and obligations, such as favoring a loyal customer with predictable revenue over a smaller account requiring investment.

When reviewing recommendations, it is crucial to inquire about the assumptions that significantly influenced the outcome, rather than seeking a comforting explanation of why the result is correct. A helpful explanation should enable disagreement. Additionally, maintaining a record of the chosen objective, the alternatives rejected, and the reasons for rejection is beneficial.

This should not be overly burdensome but is essential when decisions impact someone's opportunities, significant resources, or establish guidelines that will be applied repeatedly. By leaving room for AI to help explore alternatives and recognize consequences that might otherwise be overlooked, it is vital to ensure that choices remain open long enough for people to examine them before approving the next recommendation.

Rewritten by urgent.news; may contain errors — check the original.

Source: hackernoon.com

Read the original at hackernoon.com →

Report a problem with this story

More in AI

Homeland Secretary Assures FAA Crisis Is ‘Not Another 9/11,’ Won’t Rule Out AI Mishap or Cyber Attack

"That's what a lot of people are trying to make the comparison to, and that is not the case," Markwayne Mullin says The post Homeland Secretary Assures FAA Crisis Is ‘Not Another 9/11,’ Won’t Rule Out…

  • FAA Secretary Mullin assures ground stop is not comparable to 9/11.
  • Cause of telecommunication outage under investigation.
  • Mullin won't rule out AI mishap or cyber attack risks.

More from Saturday 14 November →