{
  "id": 9768844,
  "title": "Graphite: making two LLMs draw their own comparison",
  "url": "https://urgent.news/2026/09/25/graphite-making-two-llms-draw-their-own-comparison",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-25T13:12:22.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/harishkotra/graphite-making-two-llms-draw-their-own-comparison-13p1"
  },
  "original_language": "en",
  "account": "Graphite is an application that pits two language models against each other by giving them the same CSV data and instructing them to generate Python code using pandas and matplotlib to create the most insightful chart of the dataset. The code produced by the models is then executed in a sandbox, and the resulting PNGs are compared side-by-side along with the code, any traceback errors, and measured properties of the charts. A third model then scores the charts, determining which one is deemed the best. This process allows for a brutally objective comparison between the two models, focusing solely on the code output rather than subjective \"vibes\" or leaderboard votes.",
  "summary": "How to build an app where two models get the same CSV, write matplotlib code, get sandboxed and executed, and a third model judges the actual rendered output — measured, not vibes. The problem with AI comparisons Ask two models the same question and you get two confident answers. Which is better? Nobody knows, because nothing was built . Prompt battles compare vibes; leaderboards compare other…",
  "key_points": [
    "Graphite compares two LLMs using same CSV data",
    "Models generate Python code with pandas, matplotlib",
    "Third model scores charts based on objectivity"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}