Node.js Logistics PDF Search: Latency Caps with Budgeted Fallback Telemetry
TL;DR: Put a deadline around vector retrieval in Node.js, return keyword matches when that deadline expires, and count the switch as one low-cardinality metric. A search box that returns approximate help-center passages is more useful than one that hangs. Set the deadline from the response budget a user can tolerate, not from the vector store's p99. The observability bill for this design is…
Implement a deadline for vector retrieval in Node.js search functionality, returning keyword matches if the deadline is exceeded. Treat vector search as a bounded attempt, not the primary request owner. Use a small label set for telemetry, such as mode=vector|keyword and outcome=success|timeout|error. Do not include raw question, PDF name, customer ID, or request ID in metric labels to avoid an unbounded index.
Start with the user's end-to-end response budget, subtracting time for keyword retrieval and serialization, then validate with workload tests. The cap should be set based on the user's tolerance, not the vector store's p99 latency. Calculate retention based on event bytes and days, considering sampling strategies for a balance between retention and storage costs.
Retain a counter with two retrieval modes and three outcomes for cardinality management. Use Infrai's REST surface and API key for authenticated calls, handling non-success responses and backoff on HTTP 429 errors.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
