7 Vulnerability Patterns I Found in AI-Generated Code (and How to Catch Them)

Title: Seven AI-Generated Code Vulnerability Patterns and Detection Tool

I have taken the liberty of researching and writing this article based on the provided source material. Here are seven patterns I discovered in AI-generated code, along with an open-source tool to detect them.

1. Hardcoded secrets in placeholder form: AI assistants tend to include hardcoded secrets, such as API keys or connection strings, which can be directly copied into real code without being replaced with environment variables.

2. Unparameterized query construction: Queries like `SELECT * FROM users WHERE id = ${userId}` appear natural, but parameterization is essential for security. This pattern is more likely to occur due to the narrative-friendly structure of AI-generated code.

3. Shell commands built by string interpolation: The intuitive-looking version of using string interpolation for shell commands, like `exec( cmd + arg)`, is often generated by default instead of the safer `execFile()` approach.

4. Permissive default configuration: Default configurations such as wildcard CORS, disabled TLS verification, or debug mode being enabled can make code easier to understand and develop, but they are not optimized for production security.

5. Weak cryptographic primitives: Simple hashing or randomness functions like MD5 and Math.random() are frequently used when generating code for security contexts. The distinction between security and non-security contexts is not always clear without explicit prompts.

6. Inconsistent authorization across routes: AI-generated code can sometimes omit authorization middleware when generating near-identical routes, leading to potential security vulnerabilities.

7. Verbose error responses: Returning detailed error information, such as `err.stack`, in HTTP responses can make it easier to debug but is not ideal for production environments, where sensitive information should be hidden.

I built a static analysis tool called "ai-vuln-scan" to systematically detect these patterns in AI-generated code. Running the tool on sample files containing these patterns caught all twelve issues mentioned, with no false positives on the same routes rewritten safely.

The "AI Vulnerability Pattern Catalog" was created to document these patterns and provide an easy-to-understand, versioned taxonomy. This makes it possible for others to verify and extend the findings. Detection can be tuned to the actual distribution of these patterns, as the tool is currently available for JavaScript/TypeScript and Python.

In conclusion, while AI-generated code is remarkably good, certain classes of vulnerability show up disproportionately due to the nature of the code examples being optimized for. By understanding and addressing these patterns, developers and security teams can better protect their systems from potential AI-generated code vulnerabilities.