Improving End-to-End Tests to Reduce Flakiness: Tools & Strategies

There’s nothing worse than a test suite that fails randomly. Flaky end-to-end (E2E) tests erode confidence, slow down releases, and lead developers to ignore legitimate failures. If your team starts using the term “CI roulette,” it’s time to act.

Flaky tests are fixable—but it takes visibility, discipline, and the right tooling.

What Is a Flaky Test?

A flaky test is one that fails sometimes and passes other times, without any changes to the code. They’re usually caused by:

• Timing issues (e.g. waiting for UI elements)
• External dependencies (e.g. network APIs)
• Shared state or bad test isolation
• Race conditions in the app or the test

Finding the root cause without good visibility is tough. That’s where tooling like TestResult comes in.

Detect Flakiness with Real Data

You can’t fix what you can’t see.

• Track which tests fail intermittently across runs
• Surface failure patterns (same test, different places?)
• Monitor retries over time

TestResult gives you this visibility out of the box. It highlights tests with high variance, frequent retries, or inconsistent pass rates.

Preventing Flakiness

Avoid Hard Waits

Static waits (await page.waitForTimeout(3000)) are a top cause of flakiness. They make assumptions about timing and don’t account for real-world variance.

Instead:

• Use Playwright’s waitFor* APIs that react to real DOM changes
• Prefer assertions with built-in retries
• Monitor for resource loading explicitly if needed

Tests that wait intelligently are far more stable.

Make Tests Atomic and Isolated

Flaky tests often rely on leftover state from previous tests—data, UI elements, or user sessions.

Solutions:

• Reset the environment between tests
• Use isolated user data per run
• Clean up after each test, even on failure

Avoid shared dependencies across test files unless absolutely necessary.

Detect and Fix Race Conditions

Sometimes the app under test is the problem. Race conditions in the frontend or backend can cause occasional failures.

Things to try:

• Add logging in your app to trace timing issues
• Run tests with network throttling to simulate slow clients
• Use Playwright’s tracing to inspect flaky runs

Tools help, but you may need to debug the app itself.

Leverage Tracing and Artifacts

Flaky failures are hard to reproduce locally. Artifacts like videos, screenshots, and Playwright traces can save you hours.

• Enable tracing selectively (e.g. only on failure)
• Use consistent naming to tie logs to test IDs
• Store and surface artifacts in your analytics tool

TestResult makes it easy to associate flaky runs with traces and see patterns over time.

Retry Strategically, Not Blindly

Retries can hide flakiness instead of fixing it.

• Use retries to detect flakiness, not just mask it
• Log when a test needed a retry, not just when it passed
• Alert on retry spikes

Retries should help you improve stability, not ignore it.

Categorize and Prioritize Flaky Tests

Not all flaky tests are equally bad. Use your analytics to:

• Flag high-priority tests (e.g. login, checkout)
• Track tests that frequently waste developer time
• Automatically quarantine tests that flake repeatedly

Focus on the highest-impact fixes first.

Communicate and Document Known Flakes

A culture of transparency helps.

• Tag known flaky tests with TODOs or links to issues
• Document common flake causes
• Share reports in Slack or dashboards

This turns flakiness into a team problem, not just a test owner’s burden.

Set a Flake Budget

Just like error budgets in SRE, define what level of flakiness is acceptable.

• X% flaky test rate per week
• Y number of allowed retries
• Alert thresholds for regressions

TestResult supports this kind of tracking, so your team can be proactive instead of reactive.

Flaky E2E tests are more than annoying—they slow teams down and undermine trust. But with the right mix of tooling, patterns, and observability, you can take back control.

Start by identifying the worst offenders, improve your test discipline, and use analytics from tools like TestResult to stay ahead of regressions.