AeroLoop — The Air-Quality Agent That Keeps Its Own Ghosts

AeroLoop is an AI agent that predicts air pollution levels in London while remembering every failed prediction. Instead of discarding unsuccessful attempts, it saves them as historical "ghosts", recording the exact conditions under which each mistake occurred. This approach allows the AI to learn from its errors and improve its predictions over time.

AeroLoop operates by predicting PM2.5 levels using a combination of weather features and a linear correction model. After each prediction, it compares the result with the actual reading and then uses an open-weight large language model to analyze its past mistakes. Based on this analysis, the LLM suggests potential improvements to the prediction strategy.

These proposed parameters are then evaluated against a separate chronological holdout set that was never used during the tuning process. If the new strategy improves the model's performance, it becomes the new standard; otherwise, it is preserved as a rejected "ghost". The system uses a small, interpretable formula to make small adjustments to the base model output.

If the model is unavailable or produces invalid output, a deterministic heuristic can propose a candidate instead. AeroLoop's self-improvement mechanism relies on the LLM's ability to examine historical errors and propose strategy enhancements, while the holdout set ensures that no future information leaks into the evaluation process.

A demo of AeroLoop can be run locally using Ollama, with two main endpoints: one for generating a complete day of prediction and ground-truth data, and another for running the full analyze-propose-holdout-evaluate loop.