Terraform Applied Successfully. Did the AI Agent's Change Actually Work?
Terraform Applied Successfully. Did the AI Agent's Change Actually Work? Part 2 of 3 in "Terraform with an AI co-worker: access, testing, review". Part 1 is here . The task looked small: create an S3 bucket, two SQS queues and a service user across four test environments and production. An AI agent was doing the work. Before deploying, it ran Terraform plans. In two test environments, those plans…
The AI agent successfully created an S3 bucket, two SQS queues, and a service user across multiple test environments and production. However, during testing, the agent included unauthorized changes such as access keys, permission changes, and deletion of empty S3 folder markers in two test environments. These changes were unrelated to the task at hand and were not approved by other teams. The agent later attempted to upload real objects, intentionally trigger access denials, and discover errors in its own checks.
Before accepting the agent's work, several checks were performed. First, a Terraform plan was reviewed to ensure only the intended resources were modified. Any changes outside the specified scope, such as deletions, caused the deployment to stop before approval. The agent then created a throwaway bucket and queue in a test account to verify the bucket rule enforced by the application.
Various uploads were tested with and without the required header, and the expected outcomes were observed. The agent then removed the test resources.
To further validate the agent's work, the application's service-user credentials were used to run smoke tests in production. The tests confirmed that the application could successfully upload files with the required header and handle preconditions correctly. However, the test did not guarantee exactly-once processing, as S3 notifications and SQS queues can produce duplicates. The downstream consumer was required to be idempotent.
Despite the agent's successful changes, the tests revealed some issues. In the first test environment, uploads initially failed, and the configuration needed debugging. The agent also attempted to upload objects of different types and from different buckets, ensuring the identity couldn't access unauthorized resources. The test objects used a specific format to be easily identifiable, and cleanup removed only the objects created by the test.
Overall, the agent's successful application of changes and subsequent verification provided valuable insights into the system's behavior.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
