Pytest: Deselecting tests
Question:
With pytest, one can mark tests using a decorator
@pytest.mark.slow
def some_slow_test():
pass
Then, from the command line, one can tell pytest to skip the tests marked “slow”
pytest -k-slow
If I have an additional tag:
@pytest.mark.long
def some_long_test()
pass
I would like to be able to skip both long AND slow tests. I’ve tried this:
pytest -k-slow -k-long
and this:
pytest -k-slow,long
And neither seems to work.
At the command line, how do I tell pytest to skip both the slow AND the long tests?
Answers:
Looking through the pytest code (mark.py) and further experimentation shows the following seems to work:
pytest -k "-slow -long"
(Using the --collect-only option speeds up experimentation)
Additionally, with the recent addition of the “-m” command line option you should be able to write:
py.test -m "not (slow or long)"
IOW, the “-m” option accepts an expression which can make use of markers as boolean values (if a marker does not exist on a test function it’s value is False, if it exists, it is True).
It’s also possible to stack the mark decorators.
@pytest.mark.slow
@pytest.mark.main
def test_myfunction():
pass
I then called py.test -m "slow and main" and only the tests with both decorators were called.
py.test -m "not (slow and main)" resulted in the other tests running
If you are trying to run the tests from inside a python file, that is, you run your tests by calling
$ python testfile.py
which has contents
import pytest
pytest.main()
and you want to know how to pass the CLI flag in to pytest.main, the answer is:
pytest.main(["-m", "not slow"])
PS – yes, there are legitimate reasons to call tests this way. Pray you never have to learn them.
Is your test correctly written? Normally, tests start with test_?
But anyway, it depended on what you try to filter, you can filter those tests by name using -k "not slow and not long" or by tag using -m "not slow and not long".
Run tests by keyword expressions
-k:
This will run tests which contain names that match the given string expression (case-insensitive), which can include Python operators that use filenames, class names and function names as variables.
Run tests by marker expressions
-m:
Will run all tests which are decorated with the @pytest.mark.slow decorator.
More documentation info can be found on:
- https://docs.pytest.org/en/latest/usage.html
- Or do:
pytest --help
With pytest, one can mark tests using a decorator
@pytest.mark.slow
def some_slow_test():
pass
Then, from the command line, one can tell pytest to skip the tests marked “slow”
pytest -k-slow
If I have an additional tag:
@pytest.mark.long
def some_long_test()
pass
I would like to be able to skip both long AND slow tests. I’ve tried this:
pytest -k-slow -k-long
and this:
pytest -k-slow,long
And neither seems to work.
At the command line, how do I tell pytest to skip both the slow AND the long tests?
Looking through the pytest code (mark.py) and further experimentation shows the following seems to work:
pytest -k "-slow -long"
(Using the --collect-only option speeds up experimentation)
Additionally, with the recent addition of the “-m” command line option you should be able to write:
py.test -m "not (slow or long)"
IOW, the “-m” option accepts an expression which can make use of markers as boolean values (if a marker does not exist on a test function it’s value is False, if it exists, it is True).
It’s also possible to stack the mark decorators.
@pytest.mark.slow
@pytest.mark.main
def test_myfunction():
pass
I then called py.test -m "slow and main" and only the tests with both decorators were called.
py.test -m "not (slow and main)" resulted in the other tests running
If you are trying to run the tests from inside a python file, that is, you run your tests by calling
$ python testfile.py
which has contents
import pytest
pytest.main()
and you want to know how to pass the CLI flag in to pytest.main, the answer is:
pytest.main(["-m", "not slow"])
PS – yes, there are legitimate reasons to call tests this way. Pray you never have to learn them.
Is your test correctly written? Normally, tests start with test_?
But anyway, it depended on what you try to filter, you can filter those tests by name using -k "not slow and not long" or by tag using -m "not slow and not long".
Run tests by keyword expressions
-k:
This will run tests which contain names that match the given string expression (case-insensitive), which can include Python operators that use filenames, class names and function names as variables.
Run tests by marker expressions
-m:
Will run all tests which are decorated with the @pytest.mark.slow decorator.
More documentation info can be found on:
- https://docs.pytest.org/en/latest/usage.html
- Or do:
pytest --help