DoorDash Uses Multi Agent LLMs to Clean up 60,000 Feature Flags

DoorDash's experimentation platform manages over 60,000 feature flags across 623 repositories, generating approximately 2,300 new flags monthly. Stale flags, defined as those unmodified for 90 days, referenced in code, not archived, and not excluded, were identified in more than 1,000 instances. To streamline cleanup, DoorDash employed a multi-agent LLM system combining live experimentation data, engineer approval, isolated Git worktrees, and automated validation.

Evaluating 50 stale flags, the system generated 45 usable pull requests, averaging 13.8 minutes and costing $4.79 per cleanup, a significant reduction from the estimated one to two hours for manual cleanup. The cleanup process involved two phases: an orchestrator using Google's Agent Development Kit retrieved stale flag tickets from Jira, searched relevant repositories, and queried the experimentation platform through Model Context Protocol for metadata.

Claude Opus cleanup agents then operated in isolated Git worktrees, locating flag references, determining cleanup strategies, modifying source code and tests, and running builds, tests, JaCoCo patch coverage, and Detekt static analysis. A pull request was opened only after validation checks passed. With a one-hour timeout per agent, up to four agents could run concurrently per repository.

The evaluation resulted in 31 first-pass merges, 14 revisions, and five engineer interventions, with five flags requiring deep call chains and cross-interface parameter threading. No bugs or regressions were reported in the 50 evaluated changes. DoorDash plans to introduce confidence scoring for lower-risk cleanups and a post-cleanup code quality pass to identify issues like misleading variable names after flag removal.