Blue-green deployment that left the old environment running for weeks, doubling infrastructure cost
The deploy worked. The bill doubled. The blue-green cutover went perfectly. Traffic shifted to green, health checks passed, the team signed off, and moved on. It was one of those rare deployments that goes exactly as planned. Six weeks later, a cost anomaly surfaced in the monthly AWS review. Infrastructure spend had been running at roughly double what it should have been since the deployment…
The blue-green deployment strategy proved successful, ensuring zero downtime and instant rollback capability. However, a hidden cost emerged six weeks later, as the old environment remained running and incurring full infrastructure charges for six weeks. This anomaly occurred due to a lack of clear ownership and responsibilities post-deployment, leading to a "dangerous" open-ended rollback window.
To prevent such issues, it's advised to tag every blue environment resource with a TTL (Time To Live) during launch and set up Cost Anomaly Detection alerts for resources exceeding their TTL. This approach ensures that decommissioning is integral to the deployment process, rather than a post-deployment cleanup task.
Brief written by urgent.news from Dev.to's own syndicated text. Machine-written — may contain errors; check the original before relying on it.
