Livenerf: Has Opus 5.5 been nerfed yet?

Livenerf is a benchmark designed to detect if frontier models get worse after launch. It monitors the performance of the Claude Opus 5.5 model after its release on September 22, 2026. The benchmark runs for 30 days, with the first 10 days serving as a baseline, followed by two additional 10-day windows. The results will be published after day 20.

The benchmark measures the change in performance relative to the launch-week baseline, with a negative delta indicating worse performance. It focuses on the paired per-item score difference against the baseline, along with the output token count per sample. If the model starts thinking less, the output token count is often the first to show the change.

To run livenerf, you need Python 3.11+, uv, and a logged-in Claude Code install. The repository provides a pinned Inspect and a claudecode model provider, which wraps a hermetic claude -p call. The runner refuses to run if Claude Code changes versions, so it is crucial to pin the CLI and keep a copy of the pinned binary.

Livenerf measures Opus 5.5 as served through Claude Code on a subscription, which is the same service most nerf reports are concerned about. The benchmark also includes a control arm to account for potential infrastructure bugs or launch issues. The results will be published regardless of whether there is a change or not, ensuring transparency in the process.