I Let an AI Re-Platform My CI Pipeline. Here's What Broke.
Originally published at wostal.eu . TL;DR : I handed a CI re-platform — GitHub Actions → Argo Workflows, GitHub → Codeberg — to an AI agent. The rewrite looked correct and passed lint, but it didn't account for one thing: the runtime environment had changed completely. The old runner was an external VM; the new one is a pod inside the cluster, where Tailscale IPs aren't routable. Nobody ran the…
The author migrated their CI pipeline from GitHub Actions to Argo Workflows and from GitHub to Codeberg, trusting an AI agent to make the re-platforming. They initially assumed the rewrite looked correct and passed lint checks, but overlooked the change in runtime environment. The old runner was an external VM while the new one was a pod inside the cluster, which had different network characteristics.
This discrepancy went unnoticed because the pipeline wasn't run end-to-end to validate it in the new environment. As a result, six latent bugs emerged, including issues with the CNI configuration, Tailscale IPs, firewalls, and asymmetric routing. The most significant mistake was a missing newline in an SSH key, which caused authentication problems.
The author learned the importance of validating the entire pipeline in the new environment before trusting the migration, regardless if it was done by humans or AI. They emphasized the need for end-to-end validation to catch hidden issues that may not be apparent during a simple code review or lint check.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
