Benchmark agent harnesses side by side!

HarnessRouter serves as the foundational framework for integrating various harnesses like Codex and Claude Code into a unified agent backend. This infrastructure layer simplifies the process of running tasks, retrieving results, and seamlessly switching between harnesses through a single API. By implementing the Unified Harness Protocol (UHP), HarnessRouter provides an OpenAI Responses-compatible API, managing persistent sessions, streaming progress, files, artifacts, cancellation, and structured failures.

The platform supports multiple harness × model configurations on the same task, with each metric comparing its best and worst results. The lowest-cost and fastest configurations can vary depending on the specific task at hand.

To get started, users can run the HarnessRouter locally or integrate it into their products using starter kits. A managed Cloud Self-host Community Edition option is also available, allowing users to utilize their own provider keys on their own infrastructure. The setup process involves pulling the latest image from Docker, downloading it, and starting the container.

The console and gateway run unprivileged, while agent processes operate under the session's user. Version pinning is supported through Compose and scripted setup guides.

Upon the first launch, the enabled harness CLIs are installed. Users can access the Console by opening the provided URL or custom port and signing in with the default credentials generated during the first start. They can also set their own password in the Profile section. To run a task, users can choose a pre-configured example job from the Start menu or create their own setup by specifying the agent harness, model, skills, plugins, and tools.

The agent harness is fixed once a setup is saved, requiring the creation of a new setup for any changes.

Tasks can be sent to run using the OpenAI Responses-compatible API, selecting the harness via the metadata.harness_id. Server-sent events can be enabled for real-time task streaming, and W3C Trace Context headers can be sent to ensure correlated logging across providers. Idempotency is maintained by using unique Idempotency-Key values for each request, preventing duplicate executions. Proper handling of tokens, hashes, and sensitive information is essential to avoid accidental retyping or manipulation by the agent.