{
  "id": 12682679,
  "title": "Two 32 MB tokenizers: hunting a memory floor in a Rust proxy",
  "url": "https://urgent.news/2026/10/07/two-32-mb-tokenizers-hunting-a-memory-floor-in-a-rust-proxy",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-07T18:21:46.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/vayun/two-32-mb-tokenizers-hunting-a-memory-floor-in-a-rust-proxy-4hfl"
  },
  "original_language": "en",
  "account": "Two 32 MB tokenizers were found to be consuming excessive memory in a Rust proxy called cliproxy-rs. This proxy acts as a local server for various tools, including Codex CLI and Claude Code, forwarding requests to configured APIs. While testing on a small Linux box, the proxy's resting memory increased significantly from 26 MB to 89 MB over seven hours, with peaks reaching up to 400 MB. This type of memory leak typically starts with an upward trend that never decreases.\n\nTo investigate the issue, a soak test was conducted using a mix of Claude and Codex requests of 1 to 2 MB, with four sessions lasting an hour. The results showed that the 0.2.0 version of cliproxy-rs consumed 128 to 145 MB of memory at rest, which was deemed excessive for a proxy without caching functionality.\n\nHeaptrack analysis revealed that 67.1 MB of memory was still allocated after two batches of 100 field-mix turns, with 63.8 MB of that attributed to two o200k_base tokenizer encoders - one built by Claude count_tokens and the other by Codex count_tokens. These encoders were lazily built and never freed, leading to up to seven copies existing and consuming a total of 164 MB of memory.\n\nThe fix implemented a shared encoder per encoding, built only when a count first needs it. After implementing the fix, the resting memory dropped to 80.6 to 97.6 MB, and the peak memory usage reduced from 169.0 MB to 169.0 MB. However, the issue of memory growth under certain conditions remains unsolved, with potential causes including the field server building encoders hours apart and the impact of request shaping on memory usage.",
  "summary": "cliproxy-rs is a Rust rewrite of CLIProxyAPI. It's one local server that Codex CLI, Claude Code and other tools talk to, and it forwards to whatever you've configured: your own API keys, your own accounts. I run it on a small Linux box under real coding-agent traffic. On 0.2.0 that box showed something I didn't like. The resting memory, meaning what the process holds between requests, rose from…",
  "key_points": [
    "Two 32 MB tokenizers consume excessive memory in cliproxy-rs",
    "Memory leak starts with upward trend, never decreases",
    "Fix implements shared encoder per encoding to reduce memory"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}