The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits
Whether a language model looks demographically biased can depend on how the audit asks its question. A charitable-aid benchmark reports that the same models favor minority applicants when rating requests one at a time and penalize some when ranking side by side. We test whether that reversal generalizes to hiring, lending, and medical triage: 40,726 requests to five models, applications differing…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
