Case Study: Freeze Leaderboard Tie-Breaks Before an Agent Writes the Ranker
You should freeze leaderboard tie-break rules as executable checks before any coding agent writes the ranking function. A polished draft can still reorder equal scores in ways your support replies never promised to players. This case study follows one small ranker from a written contract, through tests, to a review gate you can rerun. The conclusion you can keep is narrow: the tests define the…
The case study explores the importance of freeze leaderboard tie-break rules before any agent writes the ranking function. The ranker must be deterministic and reject incomplete rows, raising a ValueError instead of guessing an order. The ranking rules are written in plain language before being implemented. The contract includes a decision table that maps input situations to expected results.
The provided code includes data classes for score rows, a sort_key function to generate the sorting key, and a rank_rows function to sort the rows based on the sorting key. Tests are also provided to ensure the ranking function behaves as expected, rejecting creative reorderings and only accepting the correct implementation.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
