Revisiting Joel's Test - exe.dev blog

Back in 2000, Joel Spolsky published the influential blog post, "The Joel Test: 12 Steps to Better Code," which quickly became the standard for evaluating a software team's performance. These 12 questions remain relevant even with advancements in technology, leading to the creation of the "Shelley Test," named after the coding agent and Unix shell, Mary Shelley.

The test acknowledges that peer-based code review is no longer necessary, as large language models (LLMs) now write the code, and engineers are solely responsible for it. Instead of relying on a human rubber stamp, the test advocates for using subagents with different models to perform adversarial code reviews, ensuring that the commit aligns with the intended changes.

This approach is more efficient than lengthy code review cycles and eliminates the risk of bikeshedding. Additionally, the Shelley Test emphasizes the importance of using a harness compatible with multiple models and not relying on a single family of models. Continuous deployment relies on trust in integration tests and feature flag infrastructure, making them essential components of the testing process.

The Shelley Test also highlights the significance of Athena, an agentic bot that supervises continuous deployments, reads logs, checks metrics, and writes lessons for future deploys. By using Athena, developers can quickly identify testing gaps and improve their deployment processes. Overall, the Shelley Test stresses the need for shorter cycles, continuous integration, and trust in deployment metrics to ensure the reliability of software releases in the age of LLMs.