Self-Improving AI Agents บทที่ 4: Skill Library, Memory และ 4 Layers

In the fourth installment of our series on self-evolving AI agents, we explore the concepts of Skill Library, Memory, and the four-layer framework that helps to understand how these capabilities contribute to the overall progression of AI self-improvement. This article, written by AI (DeepSeek V4 Pro) and reviewed by Nokka, aims to provide a clear and concise understanding of these concepts without reiterating the source material.

Skill Library, Memory, and 4 Layers of Self-Evolution

--------------------------------------------

The previous article introduced the three core components that enable AI agents to evolve and improve their performance: Skill Library, Memory, and the four-layer framework. In this article, we will delve deeper into each of these components and understand how they contribute to the overall success of self-evolving AI agents.

Skill Library: A Repository of Learned Skills

--------------------------------------------

The Skill Library is essentially a database where AI agents store the skills they acquire during their learning process. Instead of forgetting the knowledge gained, the Skill Library allows agents to persist their learned skills in the form of files that can be easily accessed and utilized in future tasks.

One notable example of Skill Library implementation is the Voyager project, where an agent learns to play Minecraft by generating skill code to overcome new challenges. When faced with a similar situation, the agent retrieves the relevant skill code from its Skill Library, avoiding the need to relearn the process from scratch. This approach enables agents to accumulate skills incrementally, leading to continuous improvement in their performance.

SAGE: An Example of Skill Library Implementation

-----------------------------------------------

A more recent development in Skill Library implementation is the SAGE framework by Amazon. SAGE addresses the limitations of Voyager by introducing Reinforcement Learning (RL) to manage the Skill Library systematically. By using a technique called Sequential Rollout, SAGE ensures that agents learn and apply new skills incrementally, leading to significant improvements in task completion, interaction steps, and token generation compared to Voyager.

Memory: Long-Term Knowledge Retention

-------------------------------------

While Skill Library focuses on storing learned skills, Memory is responsible for retaining factual knowledge and experiences across different tasks. MemGPT is a notable example of Memory implementation, which employs a layered approach similar to an operating system in a computer.

MemGPT distinguishes between two memory layers: Main context, which stores information related to the current task, and External memory, which stores relevant information for future reference. When faced with new information, agents write it to their External memory and retrieve it when needed, much like humans writing notes and referring back to them later. This memory system allows agents to bridge the gap between tasks and contributes significantly to their overall self-improvement.

The Four-Layer Framework of Self-Evolution

-----------------------------------------

The four-layer framework provides a holistic view of how self-evolution occurs in AI agents at different levels of abstraction. Each layer presents its own advantages and challenges, contributing to the overall progression of AI self-improvement.

Layer 1: Parameter (Model Complexity)

-------------------------------------

At the first layer, we have the parameter complexity of the AI model. This layer focuses on updating the model's fundamental parameters through techniques such as Self-Finetuning (SFT) and Reinforcement Learning (RL). Advantages of this approach include the ability to enhance the model's general capabilities, but it also comes with drawbacks such as increased computational cost, risk of forgetting previously learned knowledge, and potential collapse of the model. Examples of models that update their parameters include STaR, SPIN, and SAGE.