Automatic LLM routing in two lines of code!

Plano is an AI-native proxy server and data plane designed to make building agentic applications faster and more reliable. It abstracts away the mundane plumbing work, centralizing concerns like agent routing, orchestration, rich agentic signals, and traces for continuous improvement. Plano also provides guardrail filters for safety and moderation, as well as smart LLM routing APIs for model agility.

Developers can use any language or AI framework with Plano to deliver agents to production quickly. The platform simplifies the process by handling core delivery concerns in a unified, out-of-process data plane, freeing developers to focus on their application's core logic. Plano is built on Envoy, a widely-used software library, and leverages industry-leading LLM research.

Plano offers free hosting for its LLMs in the US-central region, allowing developers to experiment and develop agentic applications without upfront costs. For production deployment, users can either run the LLMs locally or contact Plano's team for API keys. Plano's modular building blocks handle orchestration, model management, and observability, enabling users to configure only what they need, such as edge proxying for agentic orchestration and guardrails, or LLM routing from their services.

The platform's efficiency is notable, as it utilizes lightweight LLMs (like its 4B-parameter orchestrator) instead of heavyweight frameworks or GPT-4 for routing. This results in production-grade routing at a fraction of the cost and latency. Plano traces end-to-end requests with OpenTelemetry, requiring no additional instrumentation code.

To get started with Plano, developers should follow the prerequisites guide to install the platform and set up their environment. Each request is handled by an HTTP server that implements the OpenAI-compatible chat completions endpoint, allowing developers to use any language or framework they prefer.

For further assistance, developers can join Plano's Discord server, where they can receive support and provide feedback. The platform welcomes contributions from the community, whether fixing bugs, adding new features, improving documentation, or creating tutorials. Interested parties are encouraged to visit the Contribution Guide for more details.