How I Built a NIST AI RMF-Compliant RAG System for Regulated Domains

Title: How I Built a NIST AI RMF-Compliant RAG System for Regulated Domains

The author of this article successfully implemented a production RAG (retrieval-augmented generation) system for UK arts and culture clients, ensuring compliance with the NIST AI Risk Management Framework (AI RMF). The system is specifically designed for low-risk retrieval-grounded use cases and has been deployed in an EU data residency environment, meeting GDPR requirements.

The architecture of the system consists of several key components:

1. Frontend: Streamlit Cloud

2. Vector Database: Supabase pgvector (EU-West-2)

3. Embeddings: Voyage AI (1024 dimensions)

4. LLM: Claude Haiku 4.5 (direct REST API)

5. Observability: Langfuse Integration

6. MCP server for Claude Desktop

The NIST AI RMF consists of four main functions: GOVERN, MAP, MEASURE, and MANAGE. The author documents how the system implements each of these functions.

**GOVERN**

1. Decision Authority: Solo architect with client stakeholder approval loops.

2. Data Governance (ADR-001): Use Supabase pgvector in EU-West-2 (London) due to UK public-sector cultural clients requiring UK/EU data residency for GDPR compliance. Risk of vendor dependency is mitigated through an evaluation framework and ADRs.

3. Stakeholder Roles: Developer (Lakshman), Client (UK public-sector stakeholder), Operations (future).

4. Policy: All user data stays in EU. API calls to Claude/Voyage are transient, with no data stored in the US.

5. Measurement: Langfuse audit trail logs every query's origin and destination.

**MAP**

1. Risk Inventory: Hallucination (Medium), Embedding Quality Drift (Medium), Data Drift (Low), Vendor Outage (Low), PII Leakage (Low), Prompt Injection (Low)

2. Mitigations: Prompt constraints, eval suite thresholds, monthly re-evaluation against golden questions, fallback to Ollama (local, offline), Microsoft Presidio redaction at ingestion, input validation

3. Risk Rating: LOW-RISK, as the system is retrieval-grounded and limited to a small, controlled corpus with no real-time safety-critical decisions.

**MEASURE**

1. Eval Framework (Phase 2): Built ragas-based evaluation suite with 18 golden questions and metrics for faithfulness, context precision, context recall, and answer relevancy.

2. Observability (Phase 3): Langfuse integration traces every production query with metrics such as input/output tokens, cost, latency, and trace name.

3. Cost Per Query: $0.0031 (embedding + generation), and a latency target of 2 seconds (currently ~1.2s).

**MANAGE**

1. Safeguards: Prompt engineering, error handling, rate limiting, and cost ceiling.

2. Real-world trade-offs and decision-making: ADRs detailing the choice between Supabase pgvector and self-hosted PostgreSQL, Voyage AI and Ollama, and direct Claude API access versus using Microsoft Presidio for PII redaction and CI/CD regression gates.

The author effectively demonstrates how the RAG system adheres to the NIST AI RMF guidelines while balancing trade-offs and implementing safeguards to manage identified risks.