Autoscaling Docker Containers Without Kubernetes: How Gubernator Scales CPU & GPU Workloads Automatically
When running containerized workloads, every engineering team eventually faces the scaling dilemma : Vanilla Docker / Docker Compose is lightweight, fast, and wonderfully simple to maintain—but it has zero native autoscaling . If your API traffic triples during a flash sale or your AI inference queue spikes, you must manually run docker compose up --scale api=5 . Kubernetes (K8s) provides…
When containerized workloads require scaling, developers often face challenges with traditional tools. Vanilla Docker lacks native autoscaling capabilities, while Kubernetes provides the Horizontal Pod Autoscaler (HPA) but introduces operational overhead through complex configurations. Gubernator aims to bridge this gap by offering a simple approach to horizontal autoscaling across multi-node clusters, leveraging real-time CPU and NVIDIA GPU utilization.
Gubernator's architecture consists of nodes referred to as Centurions, which can be either Managers or Workers. Multi-container applications are deployed as Legions, akin to Docker Compose stacks. The core of Gubernator's autoscaling system is an autonomous feedback loop that operates approximately every 10 to 15 seconds.
In this feedback loop, the Gubernator Watchdog triggers the autoscaler.EvaluateAndAutoscale() function. This function begins by ingesting telemetry data through Prometheus Metrics Collector, which gathers real-time CPU percentages from cAdvisor and GPU compute load from NVIDIA DCGM exporter. The evaluation process then averages the utilization metrics across healthy running task replicas for each service.
The autoscaler calculates the Desired Replicas using the formula:
Desired Replicas = ceil(Current Replicas * (Current Metric Value / Target Threshold))
This calculation is subject to guardrails and cooldown mechanisms to prevent erratic scaling behavior. Gubernator's hardware scheduler ensures that new container instances are placed on the least-loaded Centurion host, with options for GPU affinity to keep tasks close to available NVIDIA GPUs.
Declarative autoscaling policies are defined within the standard docker-compose.yml files using the gbnt.autoscaling.* labels. For CPU-based scaling, a target can be set, for example, to scale up when average CPU utilization exceeds 70%. For GPU-based scaling, particularly relevant for AI inference tasks, the target can be set based on GPU utilization, such as 80%.
When utilization drops below the target threshold, Gubernator systematically de-provisions excess containers, ensuring a graceful shutdown of tasks. The full UI control is facilitated through a Flutter Web Dashboard, providing interactive scaling badges for quick management. Every autoscaling event is logged in Gubernator's immutable Forensic Audit Log for auditing and SIEM integration.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
