{
  "id": 2161228,
  "title": "acc vs acc_norm: Why Length Bias Skews LLM Eval Scores",
  "url": "https://urgent.news/2026/08/20/acc-vs-acc-norm-why-length-bias-skews-llm-eval-scores",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-20T15:20:49.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/ji_ai/acc-vs-accnorm-why-length-bias-skews-llm-eval-scores-p6e"
  },
  "original_language": "en",
  "account": "The headline \"acc vs acc_norm: Why Length Bias Skews LLM Eval Scores\" highlights the difference between two metrics used to evaluate language models (LLMs). \"acc\" calculates the highest summed log-likelihood of a candidate continuation, while \"acc_norm\" divides that sum by the byte length of the continuation string. This difference can significantly impact the ranking of candidates, especially when answer lengths vary.",
  "summary": "Your fine-tune gains three points of acc_norm on HellaSwag and loses two points of acc . Same checkpoint, same harness, same seed. Nothing about the model's commonsense reasoning moved in two directions at once — you changed its average per-token entropy, and one of those two metrics is partly measuring how long the answer string is. This is the acc vs acc_norm problem, and it decides which…",
  "key_points": [
    "\"acc\" measures highest summed log-likelihood of candidate continuation",
    "\"accnorm\" divides sum by byte length of continuation string",
    "Length bias can skew LLM evaluation scores significantly"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}