Use case: Creating an agentic workflow from frontend to test automation

Every time the Frontend (FE) makes changes to a UI component, a series of complications often arise. Tests break, the Test Automation (TA) team is alerted, and a back-and-forth process ensues. To minimize this friction, the idea of integrating an agent into the workflow was introduced. This agent is responsible for identifying broken tests and attempting to rectify them.

Designing such an automation flow involves several considerations. One key question was timing: should the agent be triggered every time tests fail? Another was the platform on which the agentic workflow should run - Drone or GitHub Actions? Additionally, the connection between FE and TA repositories had to be addressed, as the two codebases are separate.

Lastly, the potential outcomes of the agent's actions needed to be defined: what happens if the agent cannot fix the test? What if it can? And what if it cannot reproduce the issue (in which case, the tests would pass)?

The proposed solution involves FE's CI pipeline running smoke tests. If these tests fail, an agent would be triggered in TA via a repository_dispatch event sent through the GitHub REST API. The agent would then attempt to fix the failing tests. If successful, a pull request (PR) would be created for a human to review. If this PR is merged, FE's CI pipeline would pass.

However, it's important to note that when the TA PR is merged to the main branch, FE's main branch and other branches may temporarily fail tests due to outdated code. This is a known trade-off, accepted because the process is being expedited while still maintaining control over the flow - a human will need to initiate the merge of the TA PR into FE's main branch.