Two Autoscaling Policies, Zero Coordination: Why More Signals Made Scaling Worse

A fleet of stateful worker instances behind an Auto Scaling Group was experiencing frequent size fluctuations due to two autoscaling policies with different signals, despite the intention to improve scaling performance. The initial policy utilized CPU as the scaling signal, which proved inadequate since individual instances' free slot counts could not be monitored.

To address this, a second policy was added, watching per-instance free-slot count and triggering scaling when any instance hit zero free slots. However, this led to oscillations in fleet size every 30-90 seconds, as each policy operated independently and had no way to coordinate with the other. The problem stemmed from the two policies having different opinions on capacity, each unaware of the other's existence.

The solution was to remove one of the policies and rely on a single scaling authority that accurately represented real capacity. By calculating occupancy slots in use divided by slots available using CloudWatch Metric Math, a single Target Tracking Scaling Policy was implemented, eliminating oscillations and ensuring stable fleet size.

This incident highlights the importance of using a single, meaningful metric to drive autoscaling decisions, as multiple independent policies can lead to contradictory actions without a shared source of truth.