Root Causes of Flaky Tests · Handling Flaky Tests and Best Practices in Selenium | Selenium

Root Causes of Flaky Tests

What Flaky Means

A flaky test passes and fails on the same code without any change. Flaky tests are worse than no tests: the team learns to ignore red builds and real bugs slip through. Google has reported that a large share of its test failures are flaky; every big team fights this.

Root cause	Typical symptom	Fix
Timing (no or wrong wait)	`NoSuchElementException`, `ElementClickInterceptedException`	explicit waits for the right condition
Fragile locators	fails after a small UI change	ids, `data-testid`, short CSS
Shared test data	passes alone, fails in parallel	unique data per test
Test order dependency	fails when run alone	each test sets up its own state
Environment	timeouts, 502 pages at night	health checks, retries for infra only
Animations / overlays	click lands on a spinner or banner	wait for invisibility of the overlay
Stale elements	`StaleElementReferenceException`	find the element again after the page re-renders

Triage Failures Automatically

Example

A small triage helper maps the exception in each failure to its likely cause, so the morning review starts with categories, not 30 stack traces:

import java.util.*;

public class Main {
    static final Map<String, String> CAUSES = new LinkedHashMap<>();
    static {
        CAUSES.put("NoSuchElementException", "locator or timing");
        CAUSES.put("TimeoutException", "slow page / wrong wait condition");
        CAUSES.put("StaleElementReferenceException", "DOM re-rendered, find again");
        CAUSES.put("ElementClickInterceptedException", "overlay or spinner in the way");
        CAUSES.put("AssertionError", "possible real bug - check first!");
    }

    static String classify(String failure) {
        return CAUSES.entrySet().stream()
                .filter(e -> failure.contains(e.getKey()))
                .map(Map.Entry::getValue).findFirst().orElse("unknown - read the log");
    }

    public static void main(String[] args) {
        List<String> failures = List.of(
                "addToCart: org.openqa.selenium.ElementClickInterceptedException: element click intercepted",
                "applyCoupon: java.lang.AssertionError: expected [1169.10] but found [1299.00]",
                "searchSort: org.openqa.selenium.StaleElementReferenceException: stale element reference",
                "checkoutUpi: org.openqa.selenium.TimeoutException: Expected condition failed",
                "orderHistory: java.net.SocketException: Connection reset");
        Map<String, List<String>> byCause = new TreeMap<>();
        for (String f : failures) {
            byCause.computeIfAbsent(classify(f), k -> new ArrayList<>()).add(f.substring(0, f.indexOf(':')));
        }
        byCause.forEach((cause, tests) -> System.out.println(cause + " -> " + tests));
    }
}

Output

DOM re-rendered, find again -> [searchSort]
overlay or spinner in the way -> [addToCart]
possible real bug - check first! -> [applyCoupon]
slow page / wrong wait condition -> [checkoutUpi]
unknown - read the log -> [orderHistory]