Direct Providers vs Portable Contracts — Ask-Your-Docs Semantic Search for SaaS RAG

When building a SaaS semantic search RAG pipeline for a property management system, prioritize a portable model contract for embeddings and chat completions, while keeping retrieval within the application. Add reranking only if evaluation shows first-pass semantic search is missing relevant passages. Store citations with immutable document versions, and ensure the application controls citations, schema, and routing decisions.

For embeddings and chat completions, focus on output contracts: provide minimum fields like category, confidence, escalation decision, and citations. Validate the structure, reject unknown categories, and confirm every citation came from the retrieved set. Ensure exactly-once behavior by using deterministic processing keys, recording document corpus version, query hash, retrieved chunk IDs, model selection, validated result, and request ID. Make the database write idempotent to avoid duplicate tasks.

Maintain a narrow schema with fields like category (enum), needs_human (boolean), and citations (array of chunk IDs). Avoid having the model propose workflow actions; instead, map validated categories to approved queues. In case of retries, only the committed triage record should be returned.

Prioritize retrieval quality over model fluency. Generate embeddings for document chunks and ticket questions, then store vectors with stable chunk IDs, source document IDs, revisions, jurisdiction, access scope, and exact text used for generation. Retrieve passages authorized for the specific jurisdiction and test retrieval separately from answer generation. Evaluate retrieval by comparing labeled passages to the initial candidate set, and only add reranking if it improves the process.

Consider chunking, metadata filters, and embedding choice when reranking doesn't work. Keep token counting before the answer request, and determine prompt length while assembling the prompt. Record cost estimates along with the audit record.