The Database I Deleted: Migrating an Agentic RAG App to AWS Serverless #3

Part 3 of 3 — bugs, numbers, and what generalizes

This part reveals the findings, costs, and unfinished verification from migrating an agentic RAG app to AWS serverless.

Two bugs discovered during migration:

1. Re-deriving the corpus found 303 chunks that could never be retrieved. The original chunk id format {repo}::{file_path}::{section} caused collisions when two chunks in one file shared a composed heading. Using a pre-existing retrieval bug, doc_id deduplication led to silently shadowing chunks, making some invisible during fusion.

2. A chunk that could not be embedded at all. Chunks exceeding 10,483 bytes, around 30,000 tokens, were problematic due to Titan Text Embeddings' 8,192 token limit. The solution involved splitting chunks on paragraph boundaries, with a fallback for unbroken tables and code blocks.

Key changes required for the migration:

1. Rate limiter adaptation: The crawler's sleep mechanism for GitHub quota handling was incompatible with Lambda. The function now raises instead of sleeping, allowing for wait states that do not consume billed duration.

2. Spend cap improvement: The original cap was a lifetime total, leading to budget exhaustion. The new implementation uses a DynamoDB conditional write to update the total spent within a specific window (UTC date), allowing for day-level resets and preventing budget exhaustion.

UI changes for the Serverless deployment:

1. Migrated from Streamlit, a WebSocket server, to a static SPA on S3 behind CloudFront. The UI became three files (index.html, styles.css, app.js) with no build step or framework, matching the rest of the serverless architecture.

2. The API remains the real contract, and React changes only on the server-side. The design focuses on two fundamentally different ways the assistant answers questions: exact metadata queries (amber) and document retrieval (teal), displayed through split bars within the user interface.