Token Forensics: A Self-Hosted Observatory for Every Model Call
Free tokens vanish quietly. A retry loop. A long batch. A forgotten job. A budget gate stops the bleeding. It does not tell you where the blood went. You need a different tool. Build a token observatory. It records every model call. It stores metadata in SQLite. It answers one question: where did my quota go? The Problem: Invisible Spending Free model tiers give you room to experiment. They…
Title: Token Forensics: A Self-Hosted Observatory for Every Model Call
A small FastAPI service has been created to record every model call, store metadata in SQLite, and answer the question of where the quota went. The project, named token-observatory, consists of three main jobs: forwarding chat requests to an OpenAI-compatible endpoint, recording call details, and exposing aggregate stats and a simple dashboard.
To set up the observatory, users need to obtain credentials from their MonkeyCode dashboard, including the API key, model base URL, and model name. These values must be exported in the shell and verified by making a successful request to the models endpoint.
Next, users should create a virtual environment, install the necessary dependencies, and set up the main file (app.py). The file includes the FastAPI app, SQLite database connection, and a function to initialize the database table. The ChatRequest model is defined using Pydantic, which specifies the structure of incoming chat requests.
The record_call function handles adding a new entry to the calls table, while the chat endpoint receives and processes incoming requests. It measures the time taken for the request to complete, sends the request to the OpenAI-compatible endpoint, and records the response data.
In summary, the token-observatory provides a simple yet powerful tool for monitoring and understanding token usage in model calls. By building this observatory, users can gain insight into where their resources are being spent and make informed decisions about their usage.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
