100+ Users, Free API Keys and ₹0 Cloud Budget: How I Built Kairos
I wanted to build a simple RAG system. That sentence aged badly. The original problem was actually pretty simple. We were organizing a departmental event called Kairos , where participants had to study company information and create pitch decks. The organizers had a lot of company reports. The participants had to read them. And the obvious solution was: Print everything. Except there was one tiny…
The story describes how the author built a simple Retrieval-Augmented Generation (RAG) system to power a company event called Kairos. The initial problem was the need to organize participants in studying company information and creating pitch decks, which would require printing a large number of pages. However, the author had a budget constraint of ₹0, so he decided to use AI to enable participants to ask questions instead.
The author outlined six main problems he faced while building the RAG system for Kairos:
1. Building a RAG system: The author initially thought implementing a RAG system would be simple, but it quickly became complex when he had to consider PDF ingestion, embeddings, vector search, retrieval, reranking, LLM generation, citations, frontend and backend development, authentication, rate limiting, logging, an admin dashboard, and scaling to support 100+ users.
2. Choosing an LLM: The author needed to decide which LLM to use for the system. He ruled out paid APIs due to the lack of budget and instead looked at free API tiers. He ended up using Gemini as the primary model and OpenRouter/Nemotron as a fallback.
3. Utilizing multiple API keys: To avoid hitting rate limits and costs associated with a single API key, the author implemented multiple API lanes. Each lane had its own quota and concurrency control. This approach allowed the backend to distribute requests across multiple API keys instead of overloading a single one.
4. Selecting a server: The author needed a server to run the AI system he had built. Instead of using cloud hosting, he decided to run the application locally and expose it through ngrok, effectively turning his laptop into the "cloud."
5. Managing high user traffic: The author anticipated that the system would be used by 100+ people simultaneously. To handle this, he implemented per-team question quotas, request logging and rate limiting, API key routing and fallback handling, concurrency controls, event timers, and monitoring/admin views. Each team was limited to 10 questions, which helped save API calls and made the event more engaging.
6. Addressing RAG inaccuracies: The author realized that the RAG system was not always providing accurate answers. To improve this, he implemented two-stage retrieval, where the first stage interpreted the question and created a retrieval plan, and the second stage searched the company knowledge base. He also maintained permission boundaries to separate internal instruction knowledge from company documents that participants could access.
Additionally, the system provided page-level citations to direct participants to the exact information they were looking for.
In the end, the author learned that constraints can lead to innovative solutions when faced with limited resources. Instead of seeking bigger servers or paid APIs, he focused on finding creative engineering solutions to overcome challenges. The final stack for the Kairos system included frontend technologies (React, Vite, TypeScript, Tailwind, PDF.js), backend (FastAPI, Uvicorn, Async REST APIs, SSE streaming), AI and retrieval (Gemini, OpenRouter/Nemo), and infrastructure (PostgreSQL, Qdrant).
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
