How to pass a fixture that returns a variable-length iterable of values to pytest.mark.parameterize?
Question:
I have a pytest fixture that produces an iterable and I would like to parameterize a test using the items in this iterable, but I cannot figure out the correct syntax.
Does anyone know how to parametrize a test using the values of a fixture? Here is some dummy code that shows my current approach:
import pytest
@pytest.fixture()
def values():
return [1, 1, 2]
@pytest.mark.parametrize('value', values)
def test_equal(value):
assert value == 1
Answers:
You have to add a conftest file, move your fixture, modify them, and then finally add one more function 🙂
First file:
# content of the test_<your_filename>.py
def test_equal(values):
assert values == 1
Second file:
# content of the conftest.py
def pytest_generate_tests(metafunc):
if "values" in metafunc.fixturenames:
metafunc.parametrize("values", [1, 2, 3], indirect=True)
@pytest.fixture
def values(request):
return request.param
request in the fixture is _pytest.fixtures.SubRequest instance. And param is parameter from the list in pytest_generate_tests.
And the test output:
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /mnt/d/temp
collected 3 items
test_aa.py .FF [100%]
================================ FAILURES ================================
______________________________ test_equal[2] _____________________________
def test_equal(values):
> assert values == 1
E assert 2 == 1
test_aa.py:3: AssertionError
______________________________ test_equal[3] _____________________________
values = 3
def test_equal(values):
> assert values == 1
E assert 3 == 1
test_aa.py:3: AssertionError
======================== short test summary info ========================
FAILED test_aa.py::test_equal[2] - assert 2 == 1
FAILED test_aa.py::test_equal[3] - assert 3 == 1
====================== 2 failed, 1 passed in 0.97s ======================
More details and examples you find in this part of pytest documentation.
The short answer is pytest doesn’t support passing fixtures to parametrize.
The out-of-the-box solution provided by pytest is to either use indirect parametrization or define your own parametrization scheme using pytest_generate_tests as described in How can I pass fixtures to pytest.mark.parameterize?
These are the workarounds I’ve used before to solve this problem.
Option 1: Separate function for generating values
from typing import Iterator, List
import pytest
def generate_values() -> Iterator[str]:
# ... some computationally-intensive operation ...
all_possible_values = [1, 2, 3]
for value in all_possible_values:
yield value
@pytest.fixture()
def values() -> List[str]:
return list(generate_values())
def test_all_values(values):
assert len(values) > 5
@pytest.mark.parametrize("value", generate_values())
def test_one_value_at_a_time(value: int):
assert value == 999
$ pytest -vv tests
...
========================================================== short test summary info ===========================================================
FAILED tests/test_main.py::test_all_values - assert 3 > 5
FAILED tests/test_main.py::test_one_value_at_a_time[1] - assert 1 == 999
FAILED tests/test_main.py::test_one_value_at_a_time[2] - assert 2 == 999
FAILED tests/test_main.py::test_one_value_at_a_time[3] - assert 3 == 999
The main change is moving the generation of the list of values to a regular, non-fixture function generate_values. If it’s a static list, then you can even forego making it a function and just define it as a regular module-level variable.
ALL_POSSIBLE_VALUES = [1, 2, 3]
Not everything always needs to be a fixture. It’s advantageous for injecting the test data into functions, yes, but it doesn’t mean you can’t use regular Python functions and variables. The only problem with this solution is if generating the list of values depends on other fixtures, i.e. reusable fixtures. In that case, you would have to re-define those as well.
I’ve kept the values fixture here for tests where you do need to get all the possible values as a list, like in test_all_values.
If this list of values is going to be used for multiple other tests, instead of decorating with parametrize for each one, you can do that in the pytest_generate_tests hook.
def pytest_generate_tests(metafunc: pytest.Metafunc):
if "value" in metafunc.fixturenames:
metafunc.parametrize("value", generate_values())
def test_one_value_at_a_time(value: int):
assert value == 999
This option avoids a lot of duplication and you can then even change generate_values to whatever or however you need it to be independently of the tests and the testing framework.
Option 2: Use indirect and let fixture return 1 value at a time
If it’s possible to know the length of the list of values beforehand (as in before running the tests), you can then use parametrize‘s indirect= and then let the fixture only return one value at a time.
# Set/Defined by some env/test configuration?
MAX_SUPPORTED_VALUES = 5
@pytest.fixture
def value(request: pytest.FixtureRequest) -> int:
all_possible_values = [1, 2, 3, 4, 5]
selected_index = request.param
return all_possible_values[selected_index]
@pytest.mark.parametrize("value", range(MAX_SUPPORTED_VALUES), indirect=True)
def test_one_value_at_a_time(value: int):
assert value == 999
The main change is letting the fixture accept a parameter (an index), and then returning the value at that index. (Also renaming the fixture from values to value to match the return value). Then, in the test, you use indirect=True and then pass a range of indices, which is passed to the fixture as request.param.
Again, this only works if you know at least the length of the list of values.
Also, again, instead of applying the parametrize decorator for each test that uses this fixture, you can use pytest_generate_tests:
def pytest_generate_tests(metafunc: pytest.Metafunc):
if "value" in metafunc.fixturenames:
metafunc.parametrize("value", range(MAX_SUPPORTED_VALUES), indirect=True)
def test_one_value_at_a_time(value: int):
assert value == 999
I have a pytest fixture that produces an iterable and I would like to parameterize a test using the items in this iterable, but I cannot figure out the correct syntax.
Does anyone know how to parametrize a test using the values of a fixture? Here is some dummy code that shows my current approach:
import pytest
@pytest.fixture()
def values():
return [1, 1, 2]
@pytest.mark.parametrize('value', values)
def test_equal(value):
assert value == 1
You have to add a conftest file, move your fixture, modify them, and then finally add one more function 🙂
First file:
# content of the test_<your_filename>.py
def test_equal(values):
assert values == 1
Second file:
# content of the conftest.py
def pytest_generate_tests(metafunc):
if "values" in metafunc.fixturenames:
metafunc.parametrize("values", [1, 2, 3], indirect=True)
@pytest.fixture
def values(request):
return request.param
request in the fixture is _pytest.fixtures.SubRequest instance. And param is parameter from the list in pytest_generate_tests.
And the test output:
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /mnt/d/temp
collected 3 items
test_aa.py .FF [100%]
================================ FAILURES ================================
______________________________ test_equal[2] _____________________________
def test_equal(values):
> assert values == 1
E assert 2 == 1
test_aa.py:3: AssertionError
______________________________ test_equal[3] _____________________________
values = 3
def test_equal(values):
> assert values == 1
E assert 3 == 1
test_aa.py:3: AssertionError
======================== short test summary info ========================
FAILED test_aa.py::test_equal[2] - assert 2 == 1
FAILED test_aa.py::test_equal[3] - assert 3 == 1
====================== 2 failed, 1 passed in 0.97s ======================
More details and examples you find in this part of pytest documentation.
The short answer is pytest doesn’t support passing fixtures to parametrize.
The out-of-the-box solution provided by pytest is to either use indirect parametrization or define your own parametrization scheme using pytest_generate_tests as described in How can I pass fixtures to pytest.mark.parameterize?
These are the workarounds I’ve used before to solve this problem.
Option 1: Separate function for generating values
from typing import Iterator, List
import pytest
def generate_values() -> Iterator[str]:
# ... some computationally-intensive operation ...
all_possible_values = [1, 2, 3]
for value in all_possible_values:
yield value
@pytest.fixture()
def values() -> List[str]:
return list(generate_values())
def test_all_values(values):
assert len(values) > 5
@pytest.mark.parametrize("value", generate_values())
def test_one_value_at_a_time(value: int):
assert value == 999
$ pytest -vv tests
...
========================================================== short test summary info ===========================================================
FAILED tests/test_main.py::test_all_values - assert 3 > 5
FAILED tests/test_main.py::test_one_value_at_a_time[1] - assert 1 == 999
FAILED tests/test_main.py::test_one_value_at_a_time[2] - assert 2 == 999
FAILED tests/test_main.py::test_one_value_at_a_time[3] - assert 3 == 999
The main change is moving the generation of the list of values to a regular, non-fixture function generate_values. If it’s a static list, then you can even forego making it a function and just define it as a regular module-level variable.
ALL_POSSIBLE_VALUES = [1, 2, 3]
Not everything always needs to be a fixture. It’s advantageous for injecting the test data into functions, yes, but it doesn’t mean you can’t use regular Python functions and variables. The only problem with this solution is if generating the list of values depends on other fixtures, i.e. reusable fixtures. In that case, you would have to re-define those as well.
I’ve kept the values fixture here for tests where you do need to get all the possible values as a list, like in test_all_values.
If this list of values is going to be used for multiple other tests, instead of decorating with parametrize for each one, you can do that in the pytest_generate_tests hook.
def pytest_generate_tests(metafunc: pytest.Metafunc):
if "value" in metafunc.fixturenames:
metafunc.parametrize("value", generate_values())
def test_one_value_at_a_time(value: int):
assert value == 999
This option avoids a lot of duplication and you can then even change generate_values to whatever or however you need it to be independently of the tests and the testing framework.
Option 2: Use indirect and let fixture return 1 value at a time
If it’s possible to know the length of the list of values beforehand (as in before running the tests), you can then use parametrize‘s indirect= and then let the fixture only return one value at a time.
# Set/Defined by some env/test configuration?
MAX_SUPPORTED_VALUES = 5
@pytest.fixture
def value(request: pytest.FixtureRequest) -> int:
all_possible_values = [1, 2, 3, 4, 5]
selected_index = request.param
return all_possible_values[selected_index]
@pytest.mark.parametrize("value", range(MAX_SUPPORTED_VALUES), indirect=True)
def test_one_value_at_a_time(value: int):
assert value == 999
The main change is letting the fixture accept a parameter (an index), and then returning the value at that index. (Also renaming the fixture from values to value to match the return value). Then, in the test, you use indirect=True and then pass a range of indices, which is passed to the fixture as request.param.
Again, this only works if you know at least the length of the list of values.
Also, again, instead of applying the parametrize decorator for each test that uses this fixture, you can use pytest_generate_tests:
def pytest_generate_tests(metafunc: pytest.Metafunc):
if "value" in metafunc.fixturenames:
metafunc.parametrize("value", range(MAX_SUPPORTED_VALUES), indirect=True)
def test_one_value_at_a_time(value: int):
assert value == 999