Teaching LLMs to Update Beliefs for Efficient Long-Horizon Interaction

The article discusses a new framework called ABBEL, which stands for "acting through belief bottlenecks." The key points are:

1. Traditional recursive summarization, used by LLMs to manage long interaction histories, has limitations. As tasks become more complex and require longer interactions, keeping the entire history in context becomes impractical.

2. Context summarization, a heuristic approach, has been employed to improve performance, but it has drawbacks. While summary generation reduces context size, it can't bridge the performance gap compared to models using full context.

3. The article introduces ABBEL as a solution to this problem. ABBEL acts through "belief bottlenecks," where beliefs replace the full interaction history as the agent's working context.

4. Beliefs are graded to improve performance by supervising the contents of each belief state. This approach allows the model to maintain concise, interpretable contexts while minimizing performance degradation.

5. The authors argue that creating and using human simulators to generate high-quality training environments is challenging. ABBEL aims to address this by enabling models to learn to summarize effectively, even with limited and messy interaction trajectories.

In summary, ABBEL is a framework that isolates and supervises belief states in the form of natural language, allowing LLMs to more efficiently manage long-horizon interactions by updating beliefs and reducing the need for full-context summarization.