I built an Elo rating system for cat photos, and the matchmaking was harder than the maths

Elo, the rating system behind chess rankings, consists of just fifteen lines of code. The author created this system for a project called Katwars, which showcases two cat photos at a time and allows users to tap the cuter one. This single tap updates a global rating ladder without requiring any accounts or sign-ups. The system runs on Cloudflare Workers.

The author discovered two key aspects that mattered more than expected. First, compute the delta value symmetrically for both the winner and loser to ensure the total rating in the system remains constant. Rounding the delta independently for each side can lead to rounding errors and drift in the total rating. Secondly, the K-factor, which influences how much a single result can move a rating, should be adjusted based on the number of battles.

New entrants should have a higher K-factor initially to gain a rating quickly, while established cats should have a lower K-factor for stability.

Deciding which cats should face each other proved to be the challenging part of the project. Elo's convergence depends on informative matchups. Random pairings tend to result in lopsided outcomes, providing little information about a cat's true rating. Conversely, only matching cats with similar ratings can make it difficult for new entrants to be seen.

The author settled on a weighted mix of two strategies: 75% of the time, pick a random cat within 200 rating points of the seed; the remaining 25% of the time, prioritize cats with fewer than ten matches to ensure they receive exposure.

The author implemented anonymous voting without relying on user accounts to comply with product requirements. To prevent abuse, they employed a signed device cookie, signed battle tokens, and an idempotency key per vote. Additionally, the matchmaking query excluded the user's own cat to prevent obvious abuse. They designed the system to handle the typical scenario where users might reload a page excessively while ensuring that no malicious user could manipulate the system.

The author deployed the project using the Next.js framework on Cloudflare Workers via OpenNext, with Neon Postgres over HTTP for the database and R2 for image storage. They offered a few tips for future implementations, such as monitoring the bundle size to stay within the Workers' 3 MB limit, understanding that prerendering is not caching and thus can result in 404 errors, and being aware of the limit on cron triggers, using an existing daily job instead for new scheduled work.

Reflecting on their experience, the author notes that they seeded the ladder with a few hundred placeholder cats, which helped make the leaderboard appear healthy during the launch. This strategy also taught them the importance of verifying new routes against the deployed worker, not just the build, to avoid unexpected 404 errors.