pytest exit on failure only for a specific file

Question:

I’m using the pytest framework to run tests that interface with a set of test instruments. I’d like to have a set of initial tests in a single file that will check the connection and configuration of these instruments. If any of these tests fail, I’d like to abort all future tests.

In the remaining tests and test files, I’d like pytest to continue if there are any test failures so that I get a complete report of all failed tests.

For example, the test run may look something like this:

pytest test_setup.py test_series1.py test_series2.py

In this example, I’d like pytest to exit if any tests in test_setup.py fail.

I would like to invoke the test session in a single call so that session based fixtures that I have only get called once. For example, I have a fixture that will connect to a power supply and configure it for the tests.

Is there a way to tell pytest to exit on any test only in a specific file? If I use the -x option it will not continue in subsequent tests.

Ideally, I’d prefer something like a decorator that tells pytest to exit if there is a failure. However, I have not seen anything like this.

Is this possible or am I thinking about this the wrong way?

Update

Based on the answer from pL3b, my final solution looks like this:

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    if 'critical' in [mark.name for mark in item.own_markers]:
        result = outcome.get_result()

        if result.when == "call" and result.failed:
            print('FAILED')
            pytest.exit('Exiting pytest due to critical test failure', 1)

I needed to inspect the failure code in order to check if the test failed or not. Otherwise this would exit on every call.

Additionally, I needed to register my custome marker. I chose to also put this in the conftest.py file like so:

def pytest_configure(config):
    config.addinivalue_line(
            "markers", "critical: make test as critical"
    )
Asked By: ErikJ

||

Answers:

You may use following hook in your conftest.py to solve your problem:

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    yield
    if 'critical' in [mark.name for mark in item.own_markers]:
        pytest.exit('Exiting pytest')

Then just add @pytest.mark.critical decorator to desired tests/classes in test_setup.py.

Learn more about pytest hooks here so you can define desired output and so on.

Answered By: pL3b
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.