{
  "id": 10124397,
  "title": "BrighTO-Router: an open-source Rust LLM gateway for fast, self-hosted AI traffic",
  "url": "https://urgent.news/2026/09/27/brighto-router-an-open-source-rust-llm-gateway-for-fast-self-hosted",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-27T02:41:02.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/nguyn_anhnguyn_f7d9e52/brighto-router-an-open-source-rust-llm-gateway-for-fast-self-hosted-ai-traffic-ice"
  },
  "original_language": "en",
  "account": "BrighTO-Router is an open-source LLM gateway/router that has been written in the Rust programming language. Its primary objective is to provide a single self-hosted API endpoint for AI traffic, while ensuring that the critical path remains small, fast, and easy to understand. This gateway is compatible with OpenAI and can handle chat, completions, and embeddings, as well as Anthropic messages routing. It also supports local/custom OpenAI-compatible endpoints, rerank adapters, ASR/transcription proxying, and team API keys and token budget management. The system utilizes PostgreSQL for usage metadata and ledger tracking. It employs round-robin and weighted model group load balancing to distribute model traffic efficiently. The project can be easily installed using Docker. The developers behind BrighTO-Router emphasize the importance of minimizing latency and operational complexity when placing the router on the request path. To achieve this, Rust provides predictable memory usage, fast asynchronous networking, and a deployment model that fits within a single Docker image. To showcase the router's performance, the project publishes deterministic mock-backend benchmarks that can handle payloads up to 1 million tokens. These measurements report only the router overhead, not the model inference speed, ensuring that the results are repeatable and accurate. The project's source code is available on GitHub at https://github.com/thusinh1969/BrighTO_Router, and documentation can be found at https://thusinh1969.github.io/BrighTO_Router/. For users seeking a straightforward installation process, the Docker image is available at thusinh1969/brighto_airouter:v1.",
  "summary": "I built BrighTO-Router as a free open-source LLM gateway/router written in Rust. The goal is simple: give a team one self-hosted API endpoint for AI traffic, while keeping the hot path small, fast, and understandable. What it does OpenAI-compatible chat, completions, and embeddings Anthropic Messages routing Local/custom OpenAI-compatible endpoints Rerank adapters ASR/transcription proxying Team…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}