10 agents, 30 GB of RAM and one queue: how they share without clashing
For the two weeks after I published Sluice 0.8.0, I ran it every day on my two machines, both with 30 GiB of RAM, with up to 10 AI agents working at once on one of them. Sluice puts the tests, builds and heavy commands of all the AI agents on a Linux machine into one queue, with a shared CPU and memory budget, and runs each check once per git tree; I told how it started in the first article .…
In the two weeks following the release of Sluice 0.8.0, the author ran the AI agents on two machines with 30 GiB of RAM each, simultaneously handling up to 10 agents. However, this setup led to memory issues, causing multiple out-of-memory kills and stalls. The author then implemented several changes to improve resource management.
First, the queue still determines which job runs, but the kernel's memory ceilings enforce each job's share, preventing clashes. Additionally, desktop guard ensures the desktop's share of memory remains free.
Version 0.9.0 introduced a retry mechanism where each test suite remains the same, and memory admission now carries a gate's estimate for variants, waiting for memory to be freed after any kills. When the machine thrashes, only the kernel arrives in time to enforce memory limits.
Version 0.10.0 further improved the system by running every gate job (tests, linters, type checks) under a memory ceiling enforced by the kernel, even when other jobs are stalled. Each job also receives a swap ceiling, preventing jobs from starting when the desktop session is stalled on memory.
Version 0.11.0 addressed the issue of browsers moving themselves out of their designated jobs. Chromium, used for Playwright browsers, would move itself, causing its memory to be counted and limited within the job. Sluice ensured the browser's memory was limited and died with the job.
Version 0.11.0 also introduced the concept of swapping a ceiling for the desktop, ensuring each job gets its memory reserve. Memory-related environment variables, such as NODE_OPTIONS, are now included in the cache key. Moreover, agents' environments and tokens no longer propagate to other jobs.
Version 0.12.0 introduced a daemon that shouldn't inherit the environment of the process that started it. Each agent session now has its own environment, preventing unintended memory usage. These changes collectively improved resource allocation and prevented memory clashes among AI agents.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
