{
  "id": 10242348,
  "title": "How to Build a Retriever From Scratch for a Support System",
  "url": "https://urgent.news/2026/09/27/how-to-build-a-retriever-from-scratch-for-a-support-system",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-27T13:13:49.000Z",
  "source": {
    "name": "HackerNoon",
    "slug": "hackernoon",
    "url": "https://hackernoon.com/how-to-build-a-retriever-from-scratch-for-a-support-system?source=rss"
  },
  "original_language": "en",
  "account": "If you manage a bank's customer support, you must answer queries such as the terms of a family mortgage. Don't search for a framework or an ideal RAG diagram; instead, create it from the ground up. Start by answering the simplest question, and then fix each component when it fails. To answer that single inquiry, perform two tasks: locate the internal document containing the answer, and write the response using that document. This is the complete system. Focus first on the answering half. Assume the search function works correctly and returns the appropriate document. Then, construct a prompt: \"You are Mark from support. Respond to the customer's question using only the information below. If the document doesn't contain the answer, indicate that you don't know.\" Use a small open model like Qwen3-8B with vLLM, and you have completed the baseline answer. Finding the right documents is where it gets interesting. Determine whether customers ask general questions, like rates, deposit terms, or how to close an account, or if they inquire about their own money, such as a specific account or an unfinished transaction. The answer is both. This distinction creates two search paths: one for reference data (rate sheets, product terms, promotions) and another for user data (private customer accounts, transactions, loans). Begin with the reference path using simple BM25 search. Clean the text by removing punctuation, normalising word endings, and converting to lowercase. BM25, the standard keyword search method, works initially but fails when words overlap in meaning. To address this, add semantic search by encoding the query and documents as vectors and comparing them. Store these vectors in Qdrant, Chroma, or use a framework like LlamaIndex. Paraphrases and near-synonyms will then match properly, filling BM25's blind spot. Next, prepare the knowledge base through chunking. Split documents by their structure (usually paragraphs) to ensure each chunk conveys a single idea. Be mindful of token limits, as many encoders handle 512 tokens at most. Some, like BGE-M3, support up to 8k tokens, but longer chunks may blur meaning. Balance these factors when determining chunk size. Combine both ranking lists using Reciprocal Rank Fusion (RRF), a straightforward and reliable option. Remember that a document appearing in only one list may receive an inaccurate score due to insufficient fusion. Finally, address the customer's individual data. After the initial message, the system must determine whether the query concerns the customer's accounts, transactions, or loans. Implement security measures to ensure the session never accesses another customer's data. Pass the user_id through the agent's injected state and restrict every tool call to that specific user. Choose between a single agent with multiple tools or multiple agents focusing on specific tasks. A hierarchical multi-agent system can be employed if multiple prompts are required. Design each tool with care, avoiding exposure of unnecessary backend data, and create distinct tools for each function.",
  "summary": "Build a customer-support retriever from scratch: BM25 + dense (E5) + RRF + a cross-encoder, plus an identity-scoped agent path, added step by step.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}