How to properly use mock in python with unittest setUp
Question:
In my attempt to learn TDD, trying to learn unit testing and using mock with python. Slowly getting the hang of it, but unsure if I’m doing this correctly. Forewarned: I’m stucking using python 2.4 because the vendor API’s come as pre-compiled 2.4 pyc files, so I’m using mock 0.8.0 and unittest ( not unittest2 )
Given this example code in ‘mymodule.py’
import ldap
class MyCustomException(Exception):
pass
class MyClass:
def __init__(self, server, user, passwd):
self.ldap = ldap.initialize(server)
self.user = user
self.passwd = passwd
def connect(self):
try:
self.ldap.simple_bind_s(self.user, self.passwd)
except ldap.INVALID_CREDENTIALS:
# do some stuff
raise MyCustomException
Now in my test case file ‘test_myclass.py’, I want to mock the ldap object out. ldap.initialize returns the ldap.ldapobject.SimpleLDAPObject, so I figured that’d be the method I’d have to mock out.
import unittest
from ldap import INVALID_CREDENTIALS
from mock import patch, MagicMock
from mymodule import MyClass
class LDAPConnTests(unittest.TestCase):
@patch('ldap.initialize')
def setUp(self, mock_obj):
self.ldapserver = MyClass('myserver','myuser','mypass')
self.mocked_inst = mock_obj.return_value
def testRaisesMyCustomException(self):
self.mocked_inst.simple_bind_s = MagicMock()
# set our side effect to the ldap exception to raise
self.mocked_inst.simple_bind_s.side_effect = INVALID_CREDENTIALS
self.assertRaises(mymodule.MyCustomException, self.ldapserver.connect)
def testMyNextTestCase(self):
# blah blah
Leads me to a couple of questions:
- Does that look right? 🙂
- Is that the proper way to try and mock an object that gets instantiated within the class I’m testing?
- Is it ok to be calling the @patch decorator on setUp or is this going to cause weird side effects?
- Is there anyway to get mock to raise the ldap.INVALID_CREDENTIALS exception without having to import the exception into my testcase file?
- Should I be using patch.object() instead and if so, how?
Thanks.
Answers:
You can use patch() as a class decorator, not just as a function decorator. You can then pass in the mocked function as before:
@patch('mymodule.SomeClass')
class MyTest(TestCase):
def test_one(self, MockSomeClass):
self.assertIs(mymodule.SomeClass, MockSomeClass)
See: Applying the same patch to every test method (which also lists alternatives)
It makes more sense to set up the patcher this way on setUp if you want the patching to be done for all the test methods.
If you have many patches to apply and you want them to apply to things initialised in the setUp methods too try this:
def setUp(self):
self.patches = {
"sut.BaseTestRunner._acquire_slot": mock.Mock(),
"sut.GetResource": mock.Mock(spec=GetResource),
"sut.models": mock.Mock(spec=models),
"sut.DbApi": make_db_api_mock()
}
self.applied_patches = [mock.patch(patch, data) for patch, data in self.patches.items()]
[patch.apply for patch in self.applied_patches]
.
. rest of setup
.
def tearDown(self):
patch.stopall()
I’ll start by answering your questions, and then I’ll give a detailed example of how patch() and setUp() interact.
- I don’t think it looks right, see my answer to question #3 in this list for details.
- Yes, the actual call to patch looks like it should mock the object you want.
- No, you almost never want to use the
@patch() decorator on setUp(). You got lucky, because the object is created in setUp() and never gets created during the test method.
- I don’t know of any way to make a mock object raise an exception without importing that exception into your test case file.
- I don’t see any need for
patch.object() here. It just lets you patch attributes of an object instead of specifying the target as a string.
To expand on my answer to question #3, the problem is that the patch() decorator only applies while the decorated function is running. As soon as setUp() returns, the patch is removed. In your case, that works, but I bet it would confuse someone looking at this test. If you really only want the patch to happen during setUp(), I would suggest using the with statement to make it obvious that the patch is going to be removed.
The following example has two test cases. TestPatchAsDecorator shows that decorating the class will apply the patch during the test method, but not during setUp(). TestPatchInSetUp shows how you can apply the patch so that it’s in place during both setUp() and the test method. Calling self.addCleanUp() makes sure that the patch will be removed during tearDown().
import unittest
from mock import patch
@patch('__builtin__.sum', return_value=99)
class TestPatchAsDecorator(unittest.TestCase):
def setUp(self):
s = sum([1, 2, 3])
self.assertEqual(6, s)
def test_sum(self, mock_sum):
s1 = sum([1, 2, 3])
mock_sum.return_value = 42
s2 = sum([1, 2, 3])
self.assertEqual(99, s1)
self.assertEqual(42, s2)
class TestPatchInSetUp(unittest.TestCase):
def setUp(self):
patcher = patch('__builtin__.sum', return_value=99)
self.mock_sum = patcher.start()
self.addCleanup(patcher.stop)
s = sum([1, 2, 3])
self.assertEqual(99, s)
def test_sum(self):
s1 = sum([1, 2, 3])
self.mock_sum.return_value = 42
s2 = sum([1, 2, 3])
self.assertEqual(99, s1)
self.assertEqual(42, s2)
I’d like to point out a variation of the accepted answer in which a new argument is passed to the patch() decorator:
from unittest.mock import patch, Mock
MockSomeClass = Mock()
@patch('mymodule.SomeClass', new=MockSomeClass)
class MyTest(TestCase):
def test_one(self):
# Do your test here
Note that in this case, it is no longer necessary to add the second argument, MockSomeClass, to every test method, which can save a lot of code repetition.
An explanation of this can be found at https://docs.python.org/3/library/unittest.mock.html#patch:
If patch() is used as a decorator and new is omitted, the created mock is passed in as an extra argument to the decorated function.
The answers above all omit new, but it can be convenient to include it.
You can create a patched inner function and call it from setUp.
If your original setUp function is:
def setUp(self):
some_work()
Then you can patch it by changing it to:
def setUp(self):
@patch(...)
def mocked_func():
some_work()
mocked_func()
Another way to solve this is to define the patch once outside of the class then reuse it to decorate both the class and the setUp and tearDown methods, like this (reusing Don Kirkby’s example) :
import unittest
from mock import patch
my_patch = patch('__builtin__.sum', return_value=99)
@my_patch
class TestPatchAsDecorator(unittest.TestCase):
@my_patch
def setUp(self):
s = sum([1, 2, 3])
self.assertEqual(99, s)
def test_sum(self, mock_sum):
s1 = sum([1, 2, 3])
mock_sum.return_value = 42
s2 = sum([1, 2, 3])
self.assertEqual(99, s1)
self.assertEqual(42, s2)
In my attempt to learn TDD, trying to learn unit testing and using mock with python. Slowly getting the hang of it, but unsure if I’m doing this correctly. Forewarned: I’m stucking using python 2.4 because the vendor API’s come as pre-compiled 2.4 pyc files, so I’m using mock 0.8.0 and unittest ( not unittest2 )
Given this example code in ‘mymodule.py’
import ldap
class MyCustomException(Exception):
pass
class MyClass:
def __init__(self, server, user, passwd):
self.ldap = ldap.initialize(server)
self.user = user
self.passwd = passwd
def connect(self):
try:
self.ldap.simple_bind_s(self.user, self.passwd)
except ldap.INVALID_CREDENTIALS:
# do some stuff
raise MyCustomException
Now in my test case file ‘test_myclass.py’, I want to mock the ldap object out. ldap.initialize returns the ldap.ldapobject.SimpleLDAPObject, so I figured that’d be the method I’d have to mock out.
import unittest
from ldap import INVALID_CREDENTIALS
from mock import patch, MagicMock
from mymodule import MyClass
class LDAPConnTests(unittest.TestCase):
@patch('ldap.initialize')
def setUp(self, mock_obj):
self.ldapserver = MyClass('myserver','myuser','mypass')
self.mocked_inst = mock_obj.return_value
def testRaisesMyCustomException(self):
self.mocked_inst.simple_bind_s = MagicMock()
# set our side effect to the ldap exception to raise
self.mocked_inst.simple_bind_s.side_effect = INVALID_CREDENTIALS
self.assertRaises(mymodule.MyCustomException, self.ldapserver.connect)
def testMyNextTestCase(self):
# blah blah
Leads me to a couple of questions:
- Does that look right? 🙂
- Is that the proper way to try and mock an object that gets instantiated within the class I’m testing?
- Is it ok to be calling the @patch decorator on setUp or is this going to cause weird side effects?
- Is there anyway to get mock to raise the ldap.INVALID_CREDENTIALS exception without having to import the exception into my testcase file?
- Should I be using patch.object() instead and if so, how?
Thanks.
You can use patch() as a class decorator, not just as a function decorator. You can then pass in the mocked function as before:
@patch('mymodule.SomeClass')
class MyTest(TestCase):
def test_one(self, MockSomeClass):
self.assertIs(mymodule.SomeClass, MockSomeClass)
See: Applying the same patch to every test method (which also lists alternatives)
It makes more sense to set up the patcher this way on setUp if you want the patching to be done for all the test methods.
If you have many patches to apply and you want them to apply to things initialised in the setUp methods too try this:
def setUp(self):
self.patches = {
"sut.BaseTestRunner._acquire_slot": mock.Mock(),
"sut.GetResource": mock.Mock(spec=GetResource),
"sut.models": mock.Mock(spec=models),
"sut.DbApi": make_db_api_mock()
}
self.applied_patches = [mock.patch(patch, data) for patch, data in self.patches.items()]
[patch.apply for patch in self.applied_patches]
.
. rest of setup
.
def tearDown(self):
patch.stopall()
I’ll start by answering your questions, and then I’ll give a detailed example of how patch() and setUp() interact.
- I don’t think it looks right, see my answer to question #3 in this list for details.
- Yes, the actual call to patch looks like it should mock the object you want.
- No, you almost never want to use the
@patch()decorator onsetUp(). You got lucky, because the object is created insetUp()and never gets created during the test method. - I don’t know of any way to make a mock object raise an exception without importing that exception into your test case file.
- I don’t see any need for
patch.object()here. It just lets you patch attributes of an object instead of specifying the target as a string.
To expand on my answer to question #3, the problem is that the patch() decorator only applies while the decorated function is running. As soon as setUp() returns, the patch is removed. In your case, that works, but I bet it would confuse someone looking at this test. If you really only want the patch to happen during setUp(), I would suggest using the with statement to make it obvious that the patch is going to be removed.
The following example has two test cases. TestPatchAsDecorator shows that decorating the class will apply the patch during the test method, but not during setUp(). TestPatchInSetUp shows how you can apply the patch so that it’s in place during both setUp() and the test method. Calling self.addCleanUp() makes sure that the patch will be removed during tearDown().
import unittest
from mock import patch
@patch('__builtin__.sum', return_value=99)
class TestPatchAsDecorator(unittest.TestCase):
def setUp(self):
s = sum([1, 2, 3])
self.assertEqual(6, s)
def test_sum(self, mock_sum):
s1 = sum([1, 2, 3])
mock_sum.return_value = 42
s2 = sum([1, 2, 3])
self.assertEqual(99, s1)
self.assertEqual(42, s2)
class TestPatchInSetUp(unittest.TestCase):
def setUp(self):
patcher = patch('__builtin__.sum', return_value=99)
self.mock_sum = patcher.start()
self.addCleanup(patcher.stop)
s = sum([1, 2, 3])
self.assertEqual(99, s)
def test_sum(self):
s1 = sum([1, 2, 3])
self.mock_sum.return_value = 42
s2 = sum([1, 2, 3])
self.assertEqual(99, s1)
self.assertEqual(42, s2)
I’d like to point out a variation of the accepted answer in which a new argument is passed to the patch() decorator:
from unittest.mock import patch, Mock
MockSomeClass = Mock()
@patch('mymodule.SomeClass', new=MockSomeClass)
class MyTest(TestCase):
def test_one(self):
# Do your test here
Note that in this case, it is no longer necessary to add the second argument, MockSomeClass, to every test method, which can save a lot of code repetition.
An explanation of this can be found at https://docs.python.org/3/library/unittest.mock.html#patch:
If
patch()is used as a decorator and new is omitted, the created mock is passed in as an extra argument to the decorated function.
The answers above all omit new, but it can be convenient to include it.
You can create a patched inner function and call it from setUp.
If your original setUp function is:
def setUp(self):
some_work()
Then you can patch it by changing it to:
def setUp(self):
@patch(...)
def mocked_func():
some_work()
mocked_func()
Another way to solve this is to define the patch once outside of the class then reuse it to decorate both the class and the setUp and tearDown methods, like this (reusing Don Kirkby’s example) :
import unittest
from mock import patch
my_patch = patch('__builtin__.sum', return_value=99)
@my_patch
class TestPatchAsDecorator(unittest.TestCase):
@my_patch
def setUp(self):
s = sum([1, 2, 3])
self.assertEqual(99, s)
def test_sum(self, mock_sum):
s1 = sum([1, 2, 3])
mock_sum.return_value = 42
s2 = sum([1, 2, 3])
self.assertEqual(99, s1)
self.assertEqual(42, s2)