Urgent.News

What's breaking now, across thousands of outlets.

AI

LLM agent memory compared: whole chat, summary, sliding window, retrieval and host-held state on a 14B model

Six ways of remembering what an agent has done, measured on one 14B model over 600-step jobs: the whole chat, a generic summary, a summary that writes the state out, a sliding window, retrieval, and a host-held state shown fresh each step. With the host's check in the loop, all six ended with the exact state on every step they ran. They differed in cost: 0.30 s and 21.8 joules a step for the…

Six different methods of remembering an LLM agent's actions were tested on a 14B model. These methods are whole chat, summary, sliding window, retrieval, and a host-held state. The host-held state was the cheapest and fastest method, taking only 0.3034 seconds per step and 21.8 joules per step. The whole chat method stopped at 16,000 tokens after 418 steps in three jobs, while summaries and the sliding window took significantly longer to run.

The host-held state correctly identified the model's own reply 81.89% of the time, while the other methods ranged from 84.00% to 93.83%. However, the model's own step was correct 21.63% of the time when deleting a service with edges. The host's check and repair completed any missing edges, ensuring the exact state was maintained on every step.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Other reports of this story

Every outlet that reported this, and when →

Read the original at dev.to →

More in AI

Adding AI Safety with Amazon Bedrock Guardrails: Protecting Inputs and Outputs in a RAG Application

Introduction Building an AI application that can retrieve company information and generate useful responses is exciting. But getting the model to produce a relevant answer is not the only engineering…

  • Amazon Bedrock Guardrails add AI safety to RAG applications
  • Input guardrail checks prompts before retrieval and generation
  • Output guardrail evaluates generated responses before delivery

More from Wednesday 18 November →