Building a Self-Correcting AI Agent with Reflection Loops in Python

A reflection loop is a control flow pattern used to automate tasks performed by language models. The process involves generating an output, critiquing it, and then deciding whether to retry if the output fails to meet a predefined quality bar. The simplest form of a reflection loop comprises a two-step cycle: generate and critique.

The critique step typically involves a second LLM call with a distinct prompt. However, it can also be a deterministic check - for instance, a JSON parser, a unit test runner, or a schema validator. The underlying principle is that critiquing is easier than generating. An LLM often overlooks edge cases in its first pass but can correctly identify them when asked "what is wrong with this output?". This asymmetry is what makes the reflection loop pattern effective.

The following Python class encapsulates an LLM call within a reflection loop. It takes three parameters during initialization: the task prompt, the critique prompt, and a validator function. The validator function accepts a string as input and returns a tuple containing a boolean value indicating whether the output passed the quality bar and a critique string.

To execute the reflection loop, the run method is called with a user input. The method maintains a history of attempts comprised of dictionaries detailing the attempt number, the output of the task, the critique, and whether it passed the quality bar. The method iterates until it reaches a maximum number of iterations, generating a task output and validating it.

If the output passes, the method returns a dictionary containing a success flag, the output, the number of iterations, and the history. If the output fails, the method generates a critique using the critique prompt and appends it to the history.