Building a Multi-Tenant RAG in 8GB RAM: Why I Ditched Qdrant and Kafka (Phase 0)
I have 8 GB of RAM and 6 vCPUs . With those specs, I'm building and running a full production-ready RAG system. I’m not self-hosting the LLM. I’m not spinning up Qdrant. I’m not adding Kafka. Those aren't my personal preferences—they are constraints strictly dictated by the hardware. When you run a JVM alongside a Python AI worker on an 8 GB machine, every megabyte counts. Today, I’m starting to…
With 8 GB of RAM and 6 vCPUs, the reporter is building a production-ready Retrieval-Augmented Generation (RAG) system that runs on limited hardware. They are not hosting the Large Language Model (LLM) themselves and avoid using Qdrant and Kafka due to hardware constraints. The system's architecture is 100% Dockerized and includes a Java backend API, Python AI & Embeddings Service, PostgreSQL (pgvector) for storage and vector database, Redis as a cache, and a React frontend.
The choice of PostgreSQL + pgvector over Qdrant/Milvus is due to the fact that maintaining separate databases in limited memory is impractical. Redis is preferred over Kafka for asynchronous document ingestion as it does not consume significant heap memory. The current status is Phase 0, where all five services are running cleanly, and the core design is set in the repository. However, there is no live demo yet.
Moving forward, the reporter plans to document the ingestion pipeline, which will convert raw PDFs into queryable vector chunks without triggering the Linux OOM killer. They are seeking advice on handling local embedding generation within an 8 GB RAM budget while ensuring strict multi-tenant isolation in pgvector. Suggestions for Row-Level Security, schema-per-tenant, or metadata filtering are welcome in the comments.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.