Tracing 4 Missing Feature Flag Keys After Delete-Recreate Across Logistics Cost Cohorts

This brief discusses best practices for handling missing feature flag keys in a logistics experiment. When a deleted or recreated feature flag key cannot be resolved, the system should return the caller's typed default value and emit a separate evaluation event to distinguish between a 404 lookup and an intentional fallback. The recommended approach is to maintain an append-only evaluation log and perform a replay test in a Python notebook before advancing the rollout in production.

This ensures that both incident questions (why was the tenant assigned the default?) and experiment questions (which cohort incurred the AI cost?) are answered separately. The brief emphasizes the importance of preserving four outcomes for a missing key lookup: an explicit variant, a missing key, a control-plane 404, and a locally unavailable snapshot.

Each outcome should carry a unique reason, such as KEY_NOT_FOUND or REMOTE_NOT_FOUND, to facilitate observability and prevent misleading time series analysis. The event contract should include a stable definition identifier or revision when available, or configuration and deployment revisions if not. The event should contain tenant ID, experiment cohort, flag key, definition revision, evaluated value, reason, evaluation timestamp, request correlation ID, and model usage identifiers.

By following these guidelines, the logistics experiment can accurately attribute AI cost spend without exposing sensitive shipment data in the flag event.

Brief written by urgent.news from Dev.to's own syndicated text. Machine-written — may contain errors; check the original before relying on it.