Playwright Retry Budgets Need a Failure Receipt

Retries are helpful for temporary network issues in browser tests, but they become problematic when every failure results in additional attempts without documenting the issue. A retry can transform a clear defect into a green build that nobody trusts. For Playwright suites, a small retry budget and consistent failure receipt prove more useful than unlimited patience.

The aim is not to remove retries entirely, but to make each retry explain itself. Unlimited retries can cause complications when multiple systems are involved, such as the browser, application API, and email provider. A generic retry setting of 2 does not indicate which component is unstable. Moreover, retries can increase side effects.

A test that sends an invitation, creates a payment intent, or uses a one-time token should not be automatically repeatable. Before increasing retries, consider whether the operation is idempotent and whether cleanup is performed after a failed attempt. If not, the retry may be running a second experiment with different data, rather than repeating the original test.

To create a retry budget per failure class, begin with a simple classification: browser timing, external dependencies, product assertions, and test isolation. This approach treats the budget as a diagnostic decision rather than a free pass for retries. A flaky locator may need one controlled retry, while a failed business assertion should generate an immediate failure receipt.

It's essential to maintain an honest test report, where a passing retry indicates instability, not that the scenario is healthy. Capture a useful failure receipt for each attempt, containing information such as the test, attempt number, last meaningful application event, fixture ID, relevant artifacts, and whether the retry is retryable.

Redact sensitive information from the receipt, such as inbox passwords or verification tokens. A compact pattern for attaching a receipt after a failure in Playwright involves extending the test runner and using the testInfo object to create a receipt JSON. Implement a modest global retry count, with a higher value only for tests with a documented reason.

Review the receipt before examining the final assertion to identify the root cause of the failure. Classify assertion failures separately from dependency timeouts and ensure cleanup is idempotent and safe to run after partial setup.