The Tests Pass and the Code Looks Great: 6 Things I Check in AI-Generated Code Before Merging

AI-generated code often appears flawless, neatly formatted, and well-documented, which can lead to less rigorous reviews. However, the bugs that slip through are typically more subtle than syntax errors. Here are six crucial questions to ask before merging AI-generated code:

1. Are all imports and methods valid? Models can sometimes suggest nonexistent packages or methods, especially if the package existed in an older version but has since been removed. For example, the line `import { parseDuration } from time-utils-pro;` suggests a package that does not exist. It is essential to verify the existence of the package by checking the registry, the publisher, and when it was last updated.

Additionally, the security risk is significant; if a model suggests a non-existent package, anyone could later register it and introduce harmful code.

2. Does the code verify data ownership, not just user authentication? A common mistake in generated code is checking if a user is logged in but neglecting to ensure the user has the necessary permissions to perform the action. For instance, in the DELETE request for an invoice, the code checks if the user is logged in but does not verify if the user has the right to delete the specific invoice. The fix involves adding ownership to the query, such as checking that the invoice belongs to the session's user ID.

3. What happens when something fails? Generative models often rely on try/catch blocks to suppress errors, which can hide critical issues from both users and developers. For example, a saveOrder function might log the error and return a success message even if the order was not saved. To mitigate this, it is crucial to ask who will detect the failure. If the only person aware of the failure is whoever reads the logs later, the code is not effectively handling errors.

4. Is the code compatible with the actual runtime version? AI models often produce code based on older or newer versions of a framework or library, which can lead to compatibility issues. For instance, in Next.js 15, params and searchParams must be awaited as Promises, but older examples sometimes read them directly. Similarly, outdated React patterns or deprecated config options can cause problems. Always compare the code against the official documentation for your specific version to ensure compatibility.

5. How does the code handle unexpected inputs? AI-generated code is typically designed to work with the example inputs provided in the prompt. However, real-world usage may include edge cases such as empty strings, arrays, null values, or values of the wrong type. For example, the code might be vulnerable to malicious input like { email: "a@b.com", password: { $ne: null } } if no validation is performed on the input.

Always validate input shape with a schema and ensure that comparisons of sensitive data, like hashed passwords, are performed securely.

6. Does the code include more than requested? AI-generated code often goes beyond the requested functionality, introducing additional dependencies, files, or changes that were not asked for. Before merging, carefully examine the diff statistics to identify any unnecessary additions. For instance, you might have added a new dependency in package.json or created files unrelated to the original task. Reviewing the changes thoroughly can help maintain code quality and avoid unnecessary maintenance burdens.