Don't Let Codex Roam Free: 6 Guardrails I Use for AI-Assisted Coding

In this article, the author discusses the importance of setting guardrails when using AI-assisted coding tools like Codex. They share six guardrails they employ to ensure safe and effective use of AI while working on personal web applications using Python and Flask.

The first guardrail is defining what Codex is allowed to touch. The author focuses on specific tasks, such as validating input for a sales POST endpoint or handling rollback when a database save fails. They instruct Codex not to automatically fix problems outside the current scope, but to report them instead. This helps to separate discovering a problem from fixing it.

The second guardrail involves creating failing tests before modifying production code. The author suggests reproducing the dangerous condition with a test, running the test before making the fix, confirming that the test fails, making the smallest necessary change, and then confirming that the target test passes. This process helps to verify that the problem existed, the test reproduced it, and the change closed the gap.

The third guardrail is to avoid connecting Codex-driven tests directly to the production database or normal environment. The author uses a disposable in-memory SQLite database for ordinary pytest runs and creates a separate isolated PostgreSQL environment for verifying PostgreSQL-specific behavior or migrations. This helps to reduce the impact if something goes wrong during testing.

The fourth guardrail is to prevent unexpected problems from expanding the task automatically. When Codex discovers an unrelated issue, the author stops it from fixing it immediately and instead reports it as an unresolved issue. They prioritize being able to trace what changed over fixing every discovered problem immediately.

The fifth guardrail requires a human check before committing and pushing the code changes. The author uses Codex to perform various checks, such as running pytest, checking git diff, and reviewing the results. Only after these checks pass does the author give permission to commit the changes. This adds an additional layer of protection before changes are pushed to the repository.

The sixth and final guardrail involves restricting the actual permissions of Codex. The author suggests setting up the environment so that the action is difficult or impossible to perform inadvertently. They also recommend considering human errors and creating an environment where the action is challenging to execute, providing an extra layer of safety.