I Built Select: An Open-Source Platform Where AI Agents Earn Their Autonomy

AI agents are becoming increasingly powerful, but there remains a fundamental question: how can one determine if an AI agent is ready to operate autonomously? Most existing orchestration platforms enable users to configure permissions and decide upon the level of autonomy agents should possess. In contrast, Select seeks to address this issue through a novel approach. It introduces an open-source platform designed for managing teams of AI agents, constructed using Next.js, FastAPI, PostgreSQL, and Docker.

Select's core concept diverges from conventional methods by granting agents autonomy based on their performance in completed tasks rather than providing autonomy from the outset. The platform's workflow begins with an agent being assigned a task. The agent then generates a structured result. Subsequently, deterministic rules evaluate the result against a pre-defined delivery contract.

Upon successful verification, the platform records a verification receipt. This verified result is subsequently subjected to an applicable approval policy. Verified successes contribute to an increase in an agent's autonomy level, while failures result in a decrease.

One key aspect of Select is its use of deterministic verification. The reasoning behind this is that a language model should not be empowered to judge the correctness of its own output. Select ensures the separation of duties by having the agent generate the work and a separate verifier, coded independently, evaluate the result. For instance, if an agent were to generate a customer quote, the verifier could assess aspects such as quantities, catalog prices, arithmetic, and the provenance of source information.

At present, Select offers several key features including an agent organization dashboard, task management, delivery contracts, approval flows, autonomy levels, budget management, and a fictional demo company. However, it is still in its initial beta stage. The platform still requires further development and validation, particularly in terms of live provider integrations, OAuth mailboxes, and isolated coding-agent runtimes.

Developers are invited to try out Select, available on GitHub at https://github.com/Kirby54/Select. The author expresses enthusiasm for feedback and questions regarding the potential use of evidence-based autonomy for AI agents and what would be required to instill trust in such a system.