Urgent.News

What's breaking now, across thousands of outlets.

AI

I audited my AI agent stack — 92,413 tokens burned before the first message

A while back there was a thread on r/mcp: "Running 15+ MCP servers can burn 60,000 tokens before your agent reads a message." Everyone in the comments was fixing it piecemeal — lazy-loading hacks, a homemade get-tool-help tool, grepping /context output by hand. Nobody had built the diagnostic. So I did. What it measures ctxvitals launches each stdio MCP server in your config, sends a real…

I recently conducted a thorough examination of my AI agent stack, which revealed a staggering 92,413 tokens consumed even before the first message was processed. This alarming figure was brought to light during a discussion on the r/mcp subreddit, where someone mentioned that running 15 or more MCP servers could quickly drain 60,000 tokens.

Frustrated by the lack of a diagnostic tool to shed light on this issue, I decided to create one myself. The resulting program, named ctxvitals, launches each standard input MCP server configured in your system, sends an initialize command followed by a tools/list request, and meticulously counts the actual schema tokens each server injects into the context.

This process was repeated for skills, rules files, commands, and hooks across a variety of AI configurations, including Claude Code, Cursor, Windsurf, Copilot, Codex, and Devin.

The results of my investigation were disheartening. My skill descriptions alone accounted for a whopping 39,402 tokens, while each MCP server burned through an additional 15 tokens per turn. This brought the total to a disappointing 92,413 tokens, earning me a grade F in the process. What made the situation even more embarrassing was the fact that the same MCP server was configured under three different names, resulting in the same plaintext API key being billed three times.

Additionally, one server pointed to a binary that no longer existed, further exacerbating the issue.

As I shared the per-tool rankings of my AI stack on Reddit, I came across a suggestion to measure the token consumption of Claude Code's Read tool, which turned out to be a staggering 7,487 tokens. This revelation prompted me to delve deeper into the security aspects of my configuration, discovering hardcoded secrets and potential data exfiltration patterns within my skill files and environment blocks.

To address these concerns, I implemented several measures, including lazy-loading techniques, a homemade get-tool-help tool, and grep commands to manually inspect the context output. However, I soon realized that no comprehensive diagnostic tool existed to systematically audit my entire AI agent stack. Consequently, I developed ctxvitals, a lightweight Node.js program that requires no dependencies and can run on Node 18+.

ctxvitals provides a variety of options, such as generating HTML reports, piping results to JSON, skipping the server launching process, and generating a Claude skill for self-auditing. The GitHub repository for ctxvitals is available at https://github.com/princejain756/ctxvitals.

The question remains, though: what does your AI agent stack burn in tokens? I've heard reports of 92,413 tokens in my case, which is quite alarming. If you have any insights on how to optimize your stack or if you've encountered similar issues, I would be eager to hear from you.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in AI

Why Your RAG Chatbot Still Hallucinates: 5 Design Mistakes (and How to Fix Them)

You built a RAG chatbot. The documents are indexed, the vector database is humming, and the demo went well. Then a user asks about the refund policy, and the bot confidently quotes a rule that was…

  • Splitting document chunks incorrectly cuts off essential information
  • Relying solely on vector search fails to retrieve exact matches
  • Stale data in the index causes confusion in the retrieval system

More from Monday 12 October →