Playwright Email Tests Need a Cleanup Contract
Email-based end-to-end tests often look flaky for the wrong reason. A signup test creates a user, waits for a verification message, and sometimes finds an email that belongs to a previous run. The test may pass with the wrong message, fail because the inbox is full of old messages, or become slower as more CI jobs share the same mailbox. In my QA work, I treat the mailbox as a test resource with…
End-to-end email tests using email-based fixtures can be unreliable due to flakiness. This often occurs when a test creates a user, waits for a verification message, and encounters an email from a previous run. The issue arises from the test not properly handling the lifecycle of the mailbox, leading to stale messages, inbox overflow, and slower CI jobs.
To address this, a mailbox should be treated as a test resource with a clear ownership contract. This contract should include details on who owns the mailbox (using run ID and worker ID), what can be read (filtering by recipient, subject, test ID, and time), how long it can live (with a lease expiry), and who cleans it up (the test should release it, with a scheduled reaper handling abandoned leases).
The email address alone is insufficient, as older messages may have matching subjects. To ensure reliable testing, each test mailbox should have an ownership contract that is observable in Playwright. This includes recording the state of the mailbox and attempting release in teardown. Important questions to consider are: who owns the mailbox, what can be read from it, how long it can live, and who cleans it up.
By implementing these practices, email testing can become a more reliable workflow, with failures becoming more actionable and easier to diagnose.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
