Careeroza

Why Tests Are Flaky

The Timing Problem

Selenium runs your next line immediately. The web page does not: it loads data with AJAX calls, shows spinners, animates popups. If your code looks for an element before the page has added it, the test fails, even though the app is fine.

time  0 ms   click "Login"
time  5 ms   Selenium: findElement(By.id("welcome"))   -> NoSuchElementException
time 900 ms  server answers, page shows "Welcome, Priya"   (too late)

Flaky Test

A flaky test passes sometimes and fails sometimes with no code change. On a fast laptop it passes; on a busy CI server it fails. Most flaky Selenium tests are synchronization problems.

Example

A typical flaky failure in the CI log:

mvn test

Example output

[ERROR] loginShowsWelcome(com.shopease.tests.LoginTest)  Time elapsed: 1.214 s  <<< FAILURE!
org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"#welcome"}
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

Note: Other causes: shared test data, test order, animations, network. They are covered in the advance level. This module fixes timing.