Graphite: making two LLMs draw their own comparison
How to build an app where two models get the same CSV, write matplotlib code, get sandboxed and executed, and a third model judges the actual rendered output — measured, not vibes. The problem with AI comparisons Ask two models the same question and you get two confident answers. Which is better? Nobody knows, because nothing was built . Prompt battles compare vibes; leaderboards compare other…
Graphite is an application that pits two language models against each other by giving them the same CSV data and instructing them to generate Python code using pandas and matplotlib to create the most insightful chart of the dataset. The code produced by the models is then executed in a sandbox, and the resulting PNGs are compared side-by-side along with the code, any traceback errors, and measured properties of the charts.
A third model then scores the charts, determining which one is deemed the best. This process allows for a brutally objective comparison between the two models, focusing solely on the code output rather than subjective "vibes" or leaderboard votes.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
