Pytest retry logic upon one unsuccessful test

Question

In my pytest suite, I run plenty of iOS UI tests. Which comes with plenty of frustrating issues. Im experimenting with the use of a hook based retry logic. Essentially, I have a pytest_runtest_call hook, where I collect the output of a test via a yield, and do extra processing with that data. Based on the failed output of the test I would like to re-trigger the test again, including the setup and teardown of the test. Is this possible? I am trying to do this without adding any extra packages (reinventing the wheel here, but in an effort to better understand pytest logic). Here’s the idea of where I’m currently at with my hook:

def pytest_runtest_call(item):
    output = yield
    if output.excinfo:
        # The retry_logic_handler.should_try will return a bool of whether or not to retry the test
        if retry_logic_handler.should_retry(item):
            # Re-run this test including the runtest_setup & runtest_teardown 
            ...
        _validate_expected_failures(output)

I understand I will get a lot of "just write better tests" but in my case, UI testing is rather unpredictable. And the data I am collecting helps clarify why a retry is required.

So the ultimate question is, how can I add retry logic to my pytest hooks?

Asked By: Nick H

||

Source

Answer 1

Turns out this will do the trick, a different hook but the same objective.

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    report = yield
    result = report.get_result()
    if result.outcome == 'failed':
        try:
            attempt = item.runtest()
        except:
            attempt = 'failed'
        if not attempt:
            result.outcome = 'passed'

Answered By: Nick H

Pytest retry logic upon one unsuccessful test

Question:

Answers: