How three OSLabs engineers built a CLI to catch you overpaying Claude
I’m one of three developers behind PennyWyze, an open-source CLI that audits which Claude tier (Opus, Sonnet, or Haiku) is the cheapest one that still passes your quality bar. You point it at your prompt and a handful of real examples you already know the right answer to, and it tells you, in real dollars, whether you’re overpaying. How PennyWyze works You give it two things: Your prompt — the…
Three OSLabs engineers developed PennyWyze, an open-source command-line interface (CLI) tool designed to help users identify the most cost-effective Claude tier (Opus, Sonnet, or Haiku) while maintaining their desired quality standards. To use PennyWyze, users provide their prompt, a dataset containing real input-output pairs, and a minimum pass rate.
The tool then measures the performance of each model across the dataset and calculates the projected monthly cost based on the actual token usage from the Anthropic API. The cheapest tier that surpasses the specified pass rate is identified as the optimal choice. The developers emphasized the uniqueness of PennyWyze by emphasizing its focus on cost efficiency while ensuring the correct answers are maintained, rather than simply ranking models based on general performance.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
