Avoiding running top-level module code in unit test
Question:
I am attempting to unit test some Python 3 code that imports a module. Unfortunately, the way the module is written, simply importing it has unpleasant side effects, which are not important for the tests. I’m trying to use unitest.mock.patch to get around it, but not having much luck.
Here is the structure of an illustrative sample:
.
└── work
├── __init__.py
├── test_work.py
├── work.py
└── work_caller.py
__init__.py is an empty file
work.py
import os
def work_on():
path = os.getcwd()
print(f"Working on {path}")
return path
def unpleasant_side_effect():
print("I am an unpleasant side effect of importing this module")
# Note that this is called simply by importing this file
unpleasant_side_effect()
work_caller.py
from work.work import work_on
class WorkCaller:
def call_work(self):
# Do important stuff that I want to test here
# This call I don't care about in the test, but it needs to be called
work_on()
test_work.py
from unittest import TestCase, mock
from work.work_caller import WorkCaller
class TestWorkMockingModule(TestCase):
def test_workcaller(self):
with mock.patch("work.work.unpleasant_side_effect") as mocked_function:
sut = WorkCaller()
sut.call_work()
In work_caller.py I only want to test the beginning code, not the call to work_on(). When I run the test, I get the following output:
paul-> python -m unittest
I am an unpleasant side effect of importing this module
Working on /Users/paul/src/patch-test
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
I was expecting that the line I am an unpleasant side effect of importing this module would not be printed because the function unpleasant_side_effect would be mocked. Where might I be going wrong?
Answers:
The unpleasant_side_effect is run for two reasons. First because the imports are handled before the test case is started and is therefore not mocked when importing is happening. Secondly, because the mocking itself imports work.py and thus runs unpleasant_side_effect even if work_caller.py was not imported.
The import problem can be solved by mocking the module work.py itself. This can either be done globally in the test module or in the testcase itself. Here I assigned it a MagicMock, which can be imported, called etc.
test_work.py
from unittest import TestCase, mock
class TestWorkMockingModule(TestCase):
def test_workcaller(self):
import sys
sys.modules['work.work'] = mock.MagicMock()
from work.work_caller import WorkCaller
sut = WorkCaller()
sut.call_work()
The downside is that work_on is also mocked, which I am not sure whether is a problem in your case.
It is not possible to not run the entire module when it is imported, since functions and classes are also statements, thus the module execution has to finish before returning to the caller, where one want to alter the imported module.
In case you asked partially about the best practice.
You should always split your code to library used by every other code and side-effect lines. And probably eliminate side-effects by calling the side-effecting code from you def main(): But if you want to keep side-effects anyway, then you could do:
work_lib.py:
...no side-effects...
work.py
from work_lib import ...
...side-effects....
test_work.py
from work_lib import ...
Another solution is to put this line ahead of any code that you don’t want to run on import:
if __name__ == "__main__":
If the code is at the highest/outermost level of a module, the name will be "main" when running directly or will be the module name when being imported. So in your example, if you put that line ahead of your call to unpleasant_side_effect(), the function wouldn’t get called when the module is imported.
I am attempting to unit test some Python 3 code that imports a module. Unfortunately, the way the module is written, simply importing it has unpleasant side effects, which are not important for the tests. I’m trying to use unitest.mock.patch to get around it, but not having much luck.
Here is the structure of an illustrative sample:
.
└── work
├── __init__.py
├── test_work.py
├── work.py
└── work_caller.py
__init__.py is an empty file
work.py
import os
def work_on():
path = os.getcwd()
print(f"Working on {path}")
return path
def unpleasant_side_effect():
print("I am an unpleasant side effect of importing this module")
# Note that this is called simply by importing this file
unpleasant_side_effect()
work_caller.py
from work.work import work_on
class WorkCaller:
def call_work(self):
# Do important stuff that I want to test here
# This call I don't care about in the test, but it needs to be called
work_on()
test_work.py
from unittest import TestCase, mock
from work.work_caller import WorkCaller
class TestWorkMockingModule(TestCase):
def test_workcaller(self):
with mock.patch("work.work.unpleasant_side_effect") as mocked_function:
sut = WorkCaller()
sut.call_work()
In work_caller.py I only want to test the beginning code, not the call to work_on(). When I run the test, I get the following output:
paul-> python -m unittest
I am an unpleasant side effect of importing this module
Working on /Users/paul/src/patch-test
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
I was expecting that the line I am an unpleasant side effect of importing this module would not be printed because the function unpleasant_side_effect would be mocked. Where might I be going wrong?
The unpleasant_side_effect is run for two reasons. First because the imports are handled before the test case is started and is therefore not mocked when importing is happening. Secondly, because the mocking itself imports work.py and thus runs unpleasant_side_effect even if work_caller.py was not imported.
The import problem can be solved by mocking the module work.py itself. This can either be done globally in the test module or in the testcase itself. Here I assigned it a MagicMock, which can be imported, called etc.
test_work.py
from unittest import TestCase, mock
class TestWorkMockingModule(TestCase):
def test_workcaller(self):
import sys
sys.modules['work.work'] = mock.MagicMock()
from work.work_caller import WorkCaller
sut = WorkCaller()
sut.call_work()
The downside is that work_on is also mocked, which I am not sure whether is a problem in your case.
It is not possible to not run the entire module when it is imported, since functions and classes are also statements, thus the module execution has to finish before returning to the caller, where one want to alter the imported module.
In case you asked partially about the best practice.
You should always split your code to library used by every other code and side-effect lines. And probably eliminate side-effects by calling the side-effecting code from you def main(): But if you want to keep side-effects anyway, then you could do:
work_lib.py:
...no side-effects...
work.py
from work_lib import ...
...side-effects....
test_work.py
from work_lib import ...
Another solution is to put this line ahead of any code that you don’t want to run on import:
if __name__ == "__main__":
If the code is at the highest/outermost level of a module, the name will be "main" when running directly or will be the module name when being imported. So in your example, if you put that line ahead of your call to unpleasant_side_effect(), the function wouldn’t get called when the module is imported.