Handling Retries for Unstable Tests in Playwright

When implementing test automation, the most troublesome thing is probably encountering those "`unstable tests`"

Preface

When implementing test automation, the most troublesome thing is probably encountering those “unstable tests”. These tests, without any change to the code, sometimes pass and sometimes fail. This inconsistency leads to distrust of the automation suite and wastes a great deal of precious time on constant debugging.

In this article we will share a simple but effective solution for handling unstable tests in Playwright: a custom retry utility function.

What Is an Unstable Test?

Unstable tests (or “flaky tests”) refer to tests that fail intermittently not because of code defects. Even if both the application and the test code remain unchanged, their execution results may differ.

Why Do Tests Become Unstable?

There are many reasons tests become unstable; the most common include the following:

  1. Network latency: APIs or page resources load slowly, causing test timeouts.
  2. Slow server response: The backend server is overloaded or business processing takes too long, leading to an excessively long response time.
  3. Third-party dependencies: External services relied on (such as payment gateways or social media APIs) are not stable enough, working intermittently.
  4. Race conditions: The test tries to interact with an element before the application has fully rendered or updated its state.
  5. Browser differences: Different browsers handle page loading and JavaScript execution slightly differently.

A Simple Solution

Playwright has a built-in automatic retry mechanism, but it targets the entire test.

Sometimes, if we only want to retry a part of the test instead of executing it from the start, we can define a custom retry function to handle it.

The function takes an operation (a function) as a parameter and tries to execute it multiple times on failure.

Defining the Retry Decorator

Create the retry_decorator.py file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# retry_decorator.py
import asyncio
import functools
from typing import Callable

def retry_async(max_retries: int = 3, delay: float = 1.0):
    """
    A retry decorator for async functions.

    :param max_retries: Maximum number of retries, defaults to 3.
    :param delay: Delay between retries (seconds), defaults to 1.0.
    """
    def decorator(func: Callable):
        @functools.wraps(func)
        async def wrapper(*args, **kwargs):
            last_exception = None
            for attempt in range(max_retries):
                try:
                    print(f"--- Attempting {func.__name__} (Attempt {attempt + 1}/{max_retries}) ---")
                    return await func(*args, **kwargs)
                except Exception as e:
                    last_exception = e
                    print(f"Attempt {attempt + 1} failed: {e}")
                    if attempt < max_retries - 1:
                        print(f"Waiting {delay} seconds before retry...")
                        await asyncio.sleep(delay)
            
            print(f"All {max_retries} attempts failed.")
            raise last_exception
        return wrapper
    return decorator

The core idea of this retry decorator:

  1. Loop max_Retries times.
  2. Execute the passed Callable function in the try block.
  3. If it succeeds, return the result immediately.
  4. If it fails, catch the error, wait one second, then make the next attempt.
  5. If all attempts fail, raise the last error.

Using it in a test script file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# test_example_decorator.py
import pytest
from playwright.async_api import Page, expect
from retry_decorator import retry_async

# simulated page whose content changes dynamically
FLAKY_PAGE_HTML = """
<!DOCTYPE html>
<html>
<head>
    <title>Unstable Page</title>
</head>
<body>
    <h1>Test Page</h1>
    <p id="status">Loading...</p>
    <script>
        setTimeout(() => {
            document.getElementById('status').innerText = 'Done!';
        }, 1500);
    </script>
</body>
</html>
"""

@pytest.mark.asyncio
async def test_flaky_element_with_decorator(page: Page):
    await page.set_content(FLAKY_PAGE_HTML)
    
    # call the decorated function directly; the retry logic is handled transparently
    await check_status(page)
    
    print("Test passed!")


# decorate the business handling method
@retry_async(max_retries=5, delay=1)
async def check_status(page: Page):
    """This function contains an unstable assertion, now protected by the decorator"""
    status_locator = page.locator("#status")
    await expect(status_locator).to_have_text("Done!")
    print("Status check succeeded!")

Pros and Cons of the Solution

Pros:

  • Simplicity: Easy to understand and implement.
  • Control: You can precisely control which operations need retrying, instead of retrying the entire test.
  • Reusability: The decorator can be reused across multiple tests.

Cons:

  • Masking problems: Overuse may mask the real root cause of instability (such as performance issues or race conditions).
  • Increased test time: If tests fail frequently, retries significantly increase total execution time.
  • Harder debugging: When a test fails, it may be unclear which attempt failed, and the logs may become messy.

Conclusion

Through this custom retry utility, we provide a fine-grained control layer for handling unstable tests with intermittent failures in Playwright, allowing recovery when a specific operation fails without abandoning the entire test.

Of course, in actual use, you should also avoid overuse based on the real situation of the business under test. It is best treated as a temporary solution or a patch for known, hard-to-fix instability.

For unstable tests, the most fundamental approach is still to find and resolve the root cause of the instability.

Built with Hugo
Theme Stack designed by Jimmy