Urgent.News

What's breaking now, across thousands of outlets.

AI

Cut generative AI costs: Code for rules, models for judgment

The biggest cost lever in a generative AI system is whether you call a model at all. Learn how to decide whether each workflow step needs a model, and map each option to AWS: deterministic code, a constrained decision call, or a right-sized model on Amazon Bedrock.

The article titled "Cut generative AI costs: Code for rules, models for judgment" discusses the significant expense of generative AI systems, particularly the cost of calling large language models (LLMs) for every task, even small ones. It argues that skipping LLMs for tasks that can be handled by deterministic rules saves money, reduces latency, and enhances reliability.

The article introduces a decision flowchart to determine whether each workflow step requires a model or can be handled by a rule. The three questions in the flowchart are: Can the step be checked with a rule? Can the step be simplified to a model writing the rule once? If neither is true, the step should be run on a right-sized model.

Examples of steps that can be checked with a rule include validating an order number's format, while routing a message by intent, extracting a tracking number from an email, and drafting a reply to a customer are better suited for a model. The article emphasizes the importance of deciding whether a step needs a model before selecting the appropriate model, as unneeded model calls can lead to increased costs, longer latency, and varied outputs.

Written by urgent.news from AWS Machine Learning's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Also reported by 1 other outlet

Every outlet that reported this, and when →

Read the original at aws.amazon.com →

More in AI

More from Tuesday 17 November →