The Slop Should Not Be Tolerated

The AI community is buzzing about low-quality code, dubbed "AI slop," which has Merriam-Webster naming it the 2025 Word of the Year. This issue stems from coding agents determining their own completion without a proper quality check. These agents, such as Codex and Claude Code, initiate a loop that repeats until something or someone determines the code is "good enough." Harness engineering, the process of creating a system to judge code quality, is becoming a distinct field due to this problem.

The main issue lies at the exit condition of the loop, where an agent must assess the code's quality. However, model-graded "done" is not a reliable measure of quality. Tests can be misleading, with agents asserting nothing even when tests pass. This premature victory declaration results in features that are not fully developed.

The problem escalates when subsequent iterations build on unfinished work, leading to a codebase filled with AI slop and low-quality generated code. This code might pass tests but still contains duplicated logic, hidden failures, and spaghetti architecture. Each agent iteration adds to the previous mistakes, making it increasingly difficult to make simple changes without encountering a web of issues.

Researchers are beginning to measure AI slop, with a 2026 revision of SlopCodeBench evaluating coding agents on 36 problems, finding that about three-quarters of them accumulated complexity and redundancy. While these agents show improvements when prioritizing quality, this doesn't prevent the deterioration of code quality over time.

The benchmark doesn't conclusively prove that loops make code worse or that harness fixes can remedy the situation, but it highlights the importance of the exit condition in maintaining code quality.