Thinking of ACE? We Can Do It with Fewer Tokens

Both ACE and ALTK-Evolve allow an agent to learn from its own experiences when tackling complex tasks. The key difference lies in how they utilize this learned knowledge, which ultimately impacts the number of tokens required. When faced with a multi-step task that an LLM agent fails at, it is typically not due to a lack of knowledge, but rather an inability to reliably apply that knowledge.

The agent may mis-paginate an API, choose the wrong person, or return an irrelevant value. This type of learning is possible through the agent's own history and can be incorporated into its decision-making process without the need for weight updates or human labeling.

ACE and ALTK-Evolve both employ a form of agentic memory, transforming the agent's past trajectories into reusable lessons that can be accessed during inference time without any additional computational cost. Despite their similar objectives, the two systems diverge in their approaches to delivering these lessons to the agent during inference. ACE stores all its lessons in a single, continuously evolving playbook, while ALTK-Evolve keeps them in individually retrievable guidelines.

Both systems emphasize the importance of preserving the richness of the information in the lessons, rather than collapsing it into a simplified summary. ACE achieves this by maintaining a detailed playbook with a counter for each lesson, indicating how many independent episodes contributed to its creation. ALTK-Evolve, on the other hand, assigns a support count to each guideline, which represents the number of episodes that produced it.

This count is never reduced, ensuring that the system retains the record of each lesson's experience.

The delivery method of the lessons also sets these systems apart. ACE injects its entire playbook on every step of the agent's process, regardless of the model or task at hand. In contrast, ALTK-Evolve takes a more flexible approach, treating delivery as a dial that can be adjusted based on the model's capacity and the specific task requirements.

This means that ACE consistently sends the full set of lessons, while ALTK-Evolve sends only the most relevant guidelines for each task, potentially resulting in a more efficient use of tokens.