{
  "id": 7255645,
  "title": "I Tried to Learn RAG and Got Stuck on API Costs. Turns Out My CPU Could Do the Expensive Bit.",
  "url": "https://urgent.news/2026/09/14/i-tried-to-learn-rag-and-got-stuck-on-api-costs-turns-out-my-cpu",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-14T06:00:00.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/nyagah/i-tried-to-learn-rag-and-got-stuck-on-api-costs-turns-out-my-cpu-could-do-the-expensive-bit-1o6h"
  },
  "original_language": "en",
  "account": "A few months ago, the author started building a Retrieval-Augmented Generation (RAG) application to understand the underlying mechanics of the technology. They named the project simple_rag, which uploaded a PDF, answered questions about it using an LLM, and retrieved information from the document. The author initially used Gemini for both embeddings and generation, but later switched to Mistral. Both worked, but the author disliked the architecture where each text-to-embedding conversion required calling another API with its own rate limits and costs. The author felt that for a tiny side project, this was ridiculous. The project sat dormant until the author discovered fastembed, which allowed them to generate embeddings locally on their CPU without needing a GPU, heavy PyTorch install, or per-embedding bill. This change altered the architecture significantly, with the following process: PDF/EPUB → Extract text → Split text into chunks → Generate embeddings locally on CPU → Store vectors in memory → User asks question → Embed question locally → Compare with document embeddings → Retrieve most relevant chunks → Send chunks + question to LLM → Answer. The author also realized that they didn't need a vector database, as they could store embeddings as plain Python data and use NumPy to calculate cosine similarity. For an LLM, the author chose Groq's free tier, and the RAG module handled text extraction, chunking, embedding, similarity search, retrieval, and generation. The author wanted to keep the implementation simple and understandable, focusing on the core RAG logic rather than building an enterprise-scale RAG platform.",
  "summary": "A few months ago, I started building a Retrieval-Augmented Generation application because I wanted to understand what was actually happening underneath all the buzzwords. The project I built was appropriately named simple_rag . The idea was simple enough: upload a PDF, ask questions about it, and have an LLM answer using information from the document. I got it working. Then I mostly abandoned it.…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}