We Don't Merge Red: Shifting E2E Tests Left in Mobile Development

In the world of mobile development, testing is typically performed after a feature has been shipped to production. This approach often leads to bugs being discovered only after release, resulting in lost revenue and wasted time spent on debugging and fixing issues. To address this problem, the concept of Shift-Left has emerged. Shift-left moves testing earlier in the development stages, allowing for earlier detection and resolution of issues.

The article focuses on implementing Shift-left for end-to-end (e2e) automated mobile app testing, specifically using the Detox framework for React Native apps. The app is tested locally and in CI on both Android and iOS platforms. However, the project faces challenges such as long development processes, failing tests, CI queues, and developer doubts.

To overcome these challenges, the following solutions are proposed:

1. Extend the development process by properly documenting the testing setup, providing training for developers, and demonstrating test coverage and debugging techniques.

2. Ensure that the test suite in the main branch is always green (100% pass rate). Identify and fix any flaky tests, ensuring they are independent of other factors such as other tests, user actions, time, or time zones.

3. Use pull requests (PRs) to block merges if tests are red, ensuring that only passing tests are merged into the main branch.

4. Encourage developers to write tests together with the features they are implementing, ideally in the same PR where the broken test is located.

By implementing these solutions, the project can effectively shift testing left, catching bugs earlier in the development process, reducing technical debt, and improving the overall quality of the mobile application.