Pytest – Altering the outcome based on the exception error raised

Question

I was wondering how I could alter a pytest test outcome (from a fail to a skip) in the case that my error message includes a specific string.

Occasionally we get test failures using appium where the response from the appium server is a 500 error with the failure message: "An unknown server-side error occurred while processing the command." Its an issue that we need to solve, but for the meantime we want to basically say, if the test failed because of an error message similar to that, skip the test instead of failing it.

Ive considered and tried something like this:

def pytest_runtest_setup(item):
    excinfo = None
    try:
        item.obj()
    except Exception as e:
        excinfo = sys.exc_info()
    if excinfo and "An unknown server-side error occurred while processing the command." in str(excinfo[1]):
        pytest.skip("Skipping test due to error message")

And this obviously won’t work.

But I was hoping for a similar approach.

The successful answer:
In order for me to get this working @Teejay pointed out below I needed to use the runtest_call hook and assess the message there. Currently working well in my test suite!

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_call(item):
    output = yield
    if output.excinfo:
        for potential_error_message in expected_failure_messages.keys():
            if output._excinfo[1].stacktrace and potential_error_message in output._excinfo[1].stacktrace:
                pytest.skip(reason=expected_failure_messages[potential_error_message])

Asked By: Nick H

||

Source

Answer 1

Your sketched idea is close to workable. I’d skip using sys.exc_info() and just inspect the str() value of the exception, and I’d restrict the classes of exception caught to the smallest set that covers the failures you’re trying to ignore.

Something like:

try:
    item.obj()
except OSError as e: # ideally a narrower subclass or tuple of classes
    if 'An unknown server-side error occurred while processing the command.' in str(e):
        pytest.skip(f'Skipping test due to error message {e}')
    else:
        raise e

The only additional logical change I’ve made is moving the if/skip into the exception handler and re-raising if it doesn’t match the message you’re expecting.

Restricting the classes of errors matched is a best practice to avoid catching situations you didn’t intend to catch. It’s probably harmless to over-catch here where you’re inspecting the message, just a good habit to cultivate. But it might also let you identify a specific field of the exception class to check rather than just the string representation – eg OSError has the strerror attribute containing the error message from the OS, so if you’ve limited your except block to catching just those you know you’ll have that attribute available.

I chose to include the exception in the skip message, you might decide differently if they’re uninformative.

Answered By: Peter DeGlopper

Answer 2

I recommend utilizing a hook wrapper to inspect the exception raised and act accordingly

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_setup(item):
    output = yield

    if output.excinfo:
            # Additional logic to target specific error
            pytest.skip()

Answered By: Teejay Bruno

Pytest – Altering the outcome based on the exception error raised

Question:

Answers: