Flaky Test
What is Flaky Test?
Flaky tests are the primary threat to confidence in automated test suites. A test is considered flaky when it fails on one run and passes on the next with no change to the code under test. In mobile testing, the most common sources of flakiness are: implicit wait timeouts that depend on device speed, animations that haven't completed before an assertion, shared global state between tests, non-deterministic test ordering, external network calls in tests meant to be isolated, and simulator/emulator instability. Flaky tests erode trust, once engineers start ignoring a failing test as 'probably flaky', real failures are masked. The Detox framework addresses this for React Native by synchronising the test runner with the app's main thread, eliminating most timing-related flakiness.
Related terms
Flaky Test, frequently asked questions
Run your test suite multiple times (10–20 runs) on an identical codebase and flag tests that produce inconsistent outcomes. Tools like pytest-flakefinder (Python), Gradle's --rerun-tasks, or CI quarantine buckets can automate identification. In React Native, the Detox test runner reports determinism violations.
Replace implicit waits (Thread.sleep, DispatchQueue.asyncAfter) with explicit synchronisation: wait for a specific element to be visible, wait for a network request to complete, or use a framework like Detox that synchronises automatically. Isolate test state, each test should create its own data and clean up after itself.