Grouping tests in pytest: Classes vs plain functions
Question:
I’m using pytest to test my app.
pytest supports 2 approaches (that I’m aware of) of how to write tests:
- In classes:
test_feature.py -> class TestFeature -> def test_feature_sanity
- In functions:
test_feature.py -> def test_feature_sanity
Is the approach of grouping tests in a class needed? Is it allowed to backport unittest builtin module?
Which approach would you say is better and why?
Answers:
There are no strict rules regarding organizing tests into modules vs classes. It is a matter of personal preference. Initially I tried organizing tests into classes, after some time I realized I had no use for another level of organization. Nowadays I just collect test functions into modules (files).
I could see a valid use case when some tests could be logically organized into same file, but still have additional level of organization into classes (for instance to make use of class scoped fixture). But this can also be done just splitting into multiple modules.
Typically in unit testing, the object of our tests is a single function. That is, a single function gives rise to multiple tests. In reading through test code, it’s useful to have tests for a single unit be grouped together in some way (which also allows us to e.g. run all tests for a specific function), so this leaves us with two options:
- Put all tests for each function in a dedicated module
- Put all tests for each function in a class
In the first approach we would still be interested in grouping all tests related to a source module (e.g. utils.py) in some way. Now, since we are already using modules to group tests for a function, this means that we should like to use a package to group tests for a source module.
The result is one source function maps to one test module, and one source module maps to one test package.
In the second approach, we would instead have one source function map to one test class (e.g. my_function() -> TestMyFunction), and one source module map to one test module (e.g. utils.py -> test_utils.py).
It depends on the situation, perhaps, but the second approach, i.e. a class of tests for each function you are testing, seems more clear to me. Additionally, if we are testing source classes/methods, then we could simply use an inheritance hierarchy of test classes, and still retain the one source module -> one test module mapping.
Finally, another benefit to either approach over just a flat file containing tests for multiple functions, is that with classes/modules already identifying which function is being tested, you can have better names for the actual tests, e.g. test_does_x and test_handles_y instead of test_my_function_does_x and test_my_function_handles_y.
This answer presents two compelling use-cases for a TestClass in pytest:
- Joint parametrization of multiple test methods belonging to a given class.
- Reuse of test data and test logic via subclass inheritance
Joint parametrization of multiple test methods belonging to a given class.
The pytest parametrization decorator, @pytest.mark.parametrize, can be used to make inputs available to multiple methods within a class. In the code below, the inputs param1 and param2 are available to each of the methods TestGroup.test_one and TestGroup.test_two.
"""test_class_parametrization.py"""
import pytest
@pytest.mark.parametrize(
("param1", "param2"),
[
("a", "b"),
("c", "d"),
],
)
class TestGroup:
"""A class with common parameters, `param1` and `param2`."""
@pytest.fixture
def fixt(self) -> int:
"""This fixture will only be available within the scope of TestGroup"""
return 123
def test_one(self, param1: str, param2: str, fixt: int) -> None:
print("ntest_one", param1, param2, fixt)
def test_two(self, param1: str, param2: str) -> None:
print("ntest_two", param1, param2)
$ pytest -s test_class_parametrization.py
================================================================== test session starts ==================================================================
platform linux -- Python 3.8.6, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /home/jbss
plugins: pylint-0.18.0
collected 4 items
test_class_parametrization.py
test_one a b 123
.
test_one c d 123
.
test_two a b
.
test_two c d
.
=================================================================== 4 passed in 0.01s ===================================================================
Reuse of test data and test logic via subclass inheritance
I’ll use a modified version of code taken from another answer to demonstrate the usefulness of inheriting class attributes/methods from TestClass to TestSubclass:
# in file `test_example.py`
class TestClass:
VAR: int = 3
DATA: int = 4
def test_var_positive(self) -> None:
assert self.VAR >= 0
class TestSubclass(TestClass):
VAR: int = 8
def test_var_even(self) -> None:
assert self.VAR % 2 == 0
def test_data(self) -> None:
assert self.DATA == 4
Running pytest on this file causes four tests to be run:
$ pytest -v test_example.py
=========== test session starts ===========
platform linux -- Python 3.8.2, pytest-5.4.2, py-1.8.1
collected 4 items
test_example.py::TestClass::test_var_positive PASSED
test_example.py::TestSubclass::test_var_positive PASSED
test_example.py::TestSubclass::test_var_even PASSED
test_example.py::TestSubclass::test_data PASSED
In the subclass, the inherited test_var_positive method is run using the updated value self.VAR == 8, and the newly defined test_data method is run against the inherited attribute self.DATA == 4. Such method and attribute inheritance gives a flexible way to re-use or modify shared functionality between different groups of test-cases.
I’m using pytest to test my app.
pytest supports 2 approaches (that I’m aware of) of how to write tests:
- In classes:
test_feature.py -> class TestFeature -> def test_feature_sanity
- In functions:
test_feature.py -> def test_feature_sanity
Is the approach of grouping tests in a class needed? Is it allowed to backport unittest builtin module?
Which approach would you say is better and why?
There are no strict rules regarding organizing tests into modules vs classes. It is a matter of personal preference. Initially I tried organizing tests into classes, after some time I realized I had no use for another level of organization. Nowadays I just collect test functions into modules (files).
I could see a valid use case when some tests could be logically organized into same file, but still have additional level of organization into classes (for instance to make use of class scoped fixture). But this can also be done just splitting into multiple modules.
Typically in unit testing, the object of our tests is a single function. That is, a single function gives rise to multiple tests. In reading through test code, it’s useful to have tests for a single unit be grouped together in some way (which also allows us to e.g. run all tests for a specific function), so this leaves us with two options:
- Put all tests for each function in a dedicated module
- Put all tests for each function in a class
In the first approach we would still be interested in grouping all tests related to a source module (e.g. utils.py) in some way. Now, since we are already using modules to group tests for a function, this means that we should like to use a package to group tests for a source module.
The result is one source function maps to one test module, and one source module maps to one test package.
In the second approach, we would instead have one source function map to one test class (e.g. my_function() -> TestMyFunction), and one source module map to one test module (e.g. utils.py -> test_utils.py).
It depends on the situation, perhaps, but the second approach, i.e. a class of tests for each function you are testing, seems more clear to me. Additionally, if we are testing source classes/methods, then we could simply use an inheritance hierarchy of test classes, and still retain the one source module -> one test module mapping.
Finally, another benefit to either approach over just a flat file containing tests for multiple functions, is that with classes/modules already identifying which function is being tested, you can have better names for the actual tests, e.g. test_does_x and test_handles_y instead of test_my_function_does_x and test_my_function_handles_y.
This answer presents two compelling use-cases for a TestClass in pytest:
- Joint parametrization of multiple test methods belonging to a given class.
- Reuse of test data and test logic via subclass inheritance
Joint parametrization of multiple test methods belonging to a given class.
The pytest parametrization decorator, @pytest.mark.parametrize, can be used to make inputs available to multiple methods within a class. In the code below, the inputs param1 and param2 are available to each of the methods TestGroup.test_one and TestGroup.test_two.
"""test_class_parametrization.py"""
import pytest
@pytest.mark.parametrize(
("param1", "param2"),
[
("a", "b"),
("c", "d"),
],
)
class TestGroup:
"""A class with common parameters, `param1` and `param2`."""
@pytest.fixture
def fixt(self) -> int:
"""This fixture will only be available within the scope of TestGroup"""
return 123
def test_one(self, param1: str, param2: str, fixt: int) -> None:
print("ntest_one", param1, param2, fixt)
def test_two(self, param1: str, param2: str) -> None:
print("ntest_two", param1, param2)
$ pytest -s test_class_parametrization.py
================================================================== test session starts ==================================================================
platform linux -- Python 3.8.6, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /home/jbss
plugins: pylint-0.18.0
collected 4 items
test_class_parametrization.py
test_one a b 123
.
test_one c d 123
.
test_two a b
.
test_two c d
.
=================================================================== 4 passed in 0.01s ===================================================================
Reuse of test data and test logic via subclass inheritance
I’ll use a modified version of code taken from another answer to demonstrate the usefulness of inheriting class attributes/methods from TestClass to TestSubclass:
# in file `test_example.py`
class TestClass:
VAR: int = 3
DATA: int = 4
def test_var_positive(self) -> None:
assert self.VAR >= 0
class TestSubclass(TestClass):
VAR: int = 8
def test_var_even(self) -> None:
assert self.VAR % 2 == 0
def test_data(self) -> None:
assert self.DATA == 4
Running pytest on this file causes four tests to be run:
$ pytest -v test_example.py
=========== test session starts ===========
platform linux -- Python 3.8.2, pytest-5.4.2, py-1.8.1
collected 4 items
test_example.py::TestClass::test_var_positive PASSED
test_example.py::TestSubclass::test_var_positive PASSED
test_example.py::TestSubclass::test_var_even PASSED
test_example.py::TestSubclass::test_data PASSED
In the subclass, the inherited test_var_positive method is run using the updated value self.VAR == 8, and the newly defined test_data method is run against the inherited attribute self.DATA == 4. Such method and attribute inheritance gives a flexible way to re-use or modify shared functionality between different groups of test-cases.