Mocking out methods on any instance of a python class
Question:
I want to mock out methods on any instance of some class in the production code in order to facilitate testing. Is there any library in Python which could facilitate this?
Basically, I want to do the following, but in Python (the following code is Ruby, using the Mocha library):
def test_stubbing_an_instance_method_on_all_instances_of_a_class
Product.any_instance.stubs(:name).returns('stubbed_name')
assert_equal 'stubbed_name', SomeClassThatUsesProduct.get_new_product_name
end
The important thing to note from above is that I need to mock it out on the class level, since I’m actually need to mock out methods on an instance created by the thing I’m testing.
Use Case:
I have a class QueryMaker which calls a method on an instance of RemoteAPI. I want to mock out the RemoteAPI.get_data_from_remote_server method to return some constant. How do I do this inside a test without having to put a special case within the RemoteAPI code to check for what environment it’s running in.
Example of what I wanted in action:
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test.py
from a import A
from b import B
def new_foo(self):
return "New foo"
A.foo = new_foo
y = B()
if y.bar() == "New foo":
print "Success!"
Answers:
Easiest way is probably to use a class method. You really should use an instance method, but it’s a pain to create those, whereas there’s a built-in function that creates a class method. With a class method, your stub will get a reference to the class (rather than the instance) as the first argument, but since it’s a stub this probably doesn’t matter. So:
Product.name = classmethod(lambda cls: "stubbed_name")
Note that the signature of the lambda must match the signature of the method you’re replacing. Also, of course, since Python (like Ruby) is a dynamic language, there is no guarantee that someone won’t switch out your stubbed method for something else before you get your hands on the instance, though I expect you will know pretty quickly if that happens.
Edit: On further investigation, you can leave out the classmethod():
Product.name = lambda self: "stubbed_name"
I was trying to preserve the original method’s behavior as closely as possible, but it looks like it’s not actually necessary (and doesn’t preserve the behavior as I’d hoped, anyhow).
I don’t know Ruby quite well enough to tell exactly what you’re trying to do, but check out the __getattr__ method. If you define it in your class, Python will call it when code tries to access any attribute of your class that isn’t otherwise defined. Since you want it to be a method, it will need to create a method on the fly that it returns.
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... print "My number is %d" % self.number
... def __getattr__(self, attr_name):
... return lambda:"stubbed_"+attr_name
...
>>> p = Product(172)
>>> p.number
172
>>> p.name()
'stubbed_name'
>>> p.get_number()
My number is 172
>>> p.other_method()
'stubbed_other_method'
Also note that __getattr__ needs to not use any other undefined attributes of your class, or else it will be infinitely recursive, calling __getattr__ for the attribute that doesn’t exist.
... def __getattr__(self, attr_name):
... return self.x
>>> p.y
Traceback (most recent call last):
#clipped
RuntimeError: maximum recursion depth exceeded while calling a Python object
If this is something you only want to do from your test code, not the production code, then put your normal class definition in the production code file, then in the test code define the __getattr__ method (unbound), and then bind it to the class you want:
#production code
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... print "My number is %d" % self.number
...
#test code
>>> def __getattr__(self, attr):
... return lambda:"stubbed_"+attr_name
...
>>> p = Product(172)
>>> p.number
172
>>> p.name()
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
AttributeError: Product instance has no attribute 'name'
>>> Product.__getattr__ = __getattr__
>>> p.name()
'stubbed_name'
I’m not sure how this would react with a class that was already using __getattribute__ (as opposed to __getattr__, __getattribute__ is called for all attributes whether or not they exist).
If you only want to do this for specific methods that already exist, then you could do something like:
#production code
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... return self.number
...
>>> p = Product(172)
>>> p.get_number()
172
#test code
>>> def get_number(self):
... return "stub_get_number"
...
>>> Product.get_number = get_number
>>> p.get_number()
'stub_get_number'
Or if you really wanted to be elegant, you could create a wrapper function to make doing multiple methods easy:
#test code
>>> import functools
>>> def stubber(fn):
... return functools.wraps(fn)(lambda self:"stub_"+fn.__name__)
...
>>> Product.get_number = stubber(Product.get_number)
>>> p.get_number()
'stub_get_number'
Needing to mock out methods when testing is very common and there are lots of tools to help you with it in Python. The danger with “monkey patching” classes like this is that if you don’t undo it afterwards then the class has been modified for all other uses throughout your tests.
My library mock, which is one of the most popular Python mocking libraries, includes a helper called “patch” that helps you to safely patch methods or attributes on objects and classes during your tests.
The mock module is available from:
http://pypi.python.org/pypi/mock
The patch decorator can be used as a context manager or as a test decorator. You can either use it to patch out with functions yourself, or use it to automatically patch with Mock objects that are very configurable.
from a import A
from b import B
from mock import patch
def new_foo(self):
return "New foo"
with patch.object(A, 'foo', new_foo):
y = B()
if y.bar() == "New foo":
print "Success!"
This handles the unpatching for you automatically. You could get away without defining the mock function yourself:
from mock import patch
with patch.object(A, 'foo') as mock_foo:
mock_foo.return_value = "New Foo"
y = B()
if y.bar() == "New foo":
print "Success!"
Mock is the way to do it, alright.
It can be a bit tricky to make sure you’re patching the instance method on any instances created from the class.
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test.py
from a import A
from b import B
import mock
mocked_a_class = mock.Mock()
mocked_a_instance = mocked_a_class.return_value
mocked_a_instance.foo.return_value = 'New foo'
with mock.patch('b.A', mocked_a_class): # Note b.A not a.A
y = B()
if y.bar() == "New foo":
print "Success!"
Referenced in the docs, at the para starting “To configure return values on methods of instances on the patched class…”
#Orignal Class definition - path "module.Product"
class Product:
def method_A(self):
# do something
pass
def method_B(self):
self.random_attr = 1
#Test case
from module import Product
class MockedProduct(Product):
def method_B(self):
self.random_attr = 2
with mock.patch('module.Product', new=MockedProduct):
#Write test case logic here
#Now method_B function call on product class instance should return 2
#instead of 1
minimal reproducible example using pytest and monkeypatch
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test_ab.py
import pytest
from a import A
from b import B
def new_foo(self):
return "New foo"
def test_mock_instance_method(monkeypatch):
y = B()
print(y.bar())
monkeypatch.setattr(A, 'foo', new_foo)
print(y.bar())
gives you
$ pytest -rP .
============================= test session starts ==============================
platform linux -- Python 3.8.8, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /home/user/220225so
plugins: cov-3.0.0
collected 1 item
test_ab.py . [100%]
==================================== PASSES ====================================
__________________________ test_mock_instance_method ___________________________
----------------------------- Captured stdout call -----------------------------
A's foo
New foo
============================== 1 passed in 0.01s ===============================
$
I want to mock out methods on any instance of some class in the production code in order to facilitate testing. Is there any library in Python which could facilitate this?
Basically, I want to do the following, but in Python (the following code is Ruby, using the Mocha library):
def test_stubbing_an_instance_method_on_all_instances_of_a_class
Product.any_instance.stubs(:name).returns('stubbed_name')
assert_equal 'stubbed_name', SomeClassThatUsesProduct.get_new_product_name
end
The important thing to note from above is that I need to mock it out on the class level, since I’m actually need to mock out methods on an instance created by the thing I’m testing.
Use Case:
I have a class QueryMaker which calls a method on an instance of RemoteAPI. I want to mock out the RemoteAPI.get_data_from_remote_server method to return some constant. How do I do this inside a test without having to put a special case within the RemoteAPI code to check for what environment it’s running in.
Example of what I wanted in action:
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test.py
from a import A
from b import B
def new_foo(self):
return "New foo"
A.foo = new_foo
y = B()
if y.bar() == "New foo":
print "Success!"
Easiest way is probably to use a class method. You really should use an instance method, but it’s a pain to create those, whereas there’s a built-in function that creates a class method. With a class method, your stub will get a reference to the class (rather than the instance) as the first argument, but since it’s a stub this probably doesn’t matter. So:
Product.name = classmethod(lambda cls: "stubbed_name")
Note that the signature of the lambda must match the signature of the method you’re replacing. Also, of course, since Python (like Ruby) is a dynamic language, there is no guarantee that someone won’t switch out your stubbed method for something else before you get your hands on the instance, though I expect you will know pretty quickly if that happens.
Edit: On further investigation, you can leave out the classmethod():
Product.name = lambda self: "stubbed_name"
I was trying to preserve the original method’s behavior as closely as possible, but it looks like it’s not actually necessary (and doesn’t preserve the behavior as I’d hoped, anyhow).
I don’t know Ruby quite well enough to tell exactly what you’re trying to do, but check out the __getattr__ method. If you define it in your class, Python will call it when code tries to access any attribute of your class that isn’t otherwise defined. Since you want it to be a method, it will need to create a method on the fly that it returns.
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... print "My number is %d" % self.number
... def __getattr__(self, attr_name):
... return lambda:"stubbed_"+attr_name
...
>>> p = Product(172)
>>> p.number
172
>>> p.name()
'stubbed_name'
>>> p.get_number()
My number is 172
>>> p.other_method()
'stubbed_other_method'
Also note that __getattr__ needs to not use any other undefined attributes of your class, or else it will be infinitely recursive, calling __getattr__ for the attribute that doesn’t exist.
... def __getattr__(self, attr_name):
... return self.x
>>> p.y
Traceback (most recent call last):
#clipped
RuntimeError: maximum recursion depth exceeded while calling a Python object
If this is something you only want to do from your test code, not the production code, then put your normal class definition in the production code file, then in the test code define the __getattr__ method (unbound), and then bind it to the class you want:
#production code
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... print "My number is %d" % self.number
...
#test code
>>> def __getattr__(self, attr):
... return lambda:"stubbed_"+attr_name
...
>>> p = Product(172)
>>> p.number
172
>>> p.name()
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
AttributeError: Product instance has no attribute 'name'
>>> Product.__getattr__ = __getattr__
>>> p.name()
'stubbed_name'
I’m not sure how this would react with a class that was already using __getattribute__ (as opposed to __getattr__, __getattribute__ is called for all attributes whether or not they exist).
If you only want to do this for specific methods that already exist, then you could do something like:
#production code
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... return self.number
...
>>> p = Product(172)
>>> p.get_number()
172
#test code
>>> def get_number(self):
... return "stub_get_number"
...
>>> Product.get_number = get_number
>>> p.get_number()
'stub_get_number'
Or if you really wanted to be elegant, you could create a wrapper function to make doing multiple methods easy:
#test code
>>> import functools
>>> def stubber(fn):
... return functools.wraps(fn)(lambda self:"stub_"+fn.__name__)
...
>>> Product.get_number = stubber(Product.get_number)
>>> p.get_number()
'stub_get_number'
Needing to mock out methods when testing is very common and there are lots of tools to help you with it in Python. The danger with “monkey patching” classes like this is that if you don’t undo it afterwards then the class has been modified for all other uses throughout your tests.
My library mock, which is one of the most popular Python mocking libraries, includes a helper called “patch” that helps you to safely patch methods or attributes on objects and classes during your tests.
The mock module is available from:
http://pypi.python.org/pypi/mock
The patch decorator can be used as a context manager or as a test decorator. You can either use it to patch out with functions yourself, or use it to automatically patch with Mock objects that are very configurable.
from a import A
from b import B
from mock import patch
def new_foo(self):
return "New foo"
with patch.object(A, 'foo', new_foo):
y = B()
if y.bar() == "New foo":
print "Success!"
This handles the unpatching for you automatically. You could get away without defining the mock function yourself:
from mock import patch
with patch.object(A, 'foo') as mock_foo:
mock_foo.return_value = "New Foo"
y = B()
if y.bar() == "New foo":
print "Success!"
Mock is the way to do it, alright.
It can be a bit tricky to make sure you’re patching the instance method on any instances created from the class.
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test.py
from a import A
from b import B
import mock
mocked_a_class = mock.Mock()
mocked_a_instance = mocked_a_class.return_value
mocked_a_instance.foo.return_value = 'New foo'
with mock.patch('b.A', mocked_a_class): # Note b.A not a.A
y = B()
if y.bar() == "New foo":
print "Success!"
Referenced in the docs, at the para starting “To configure return values on methods of instances on the patched class…”
#Orignal Class definition - path "module.Product"
class Product:
def method_A(self):
# do something
pass
def method_B(self):
self.random_attr = 1
#Test case
from module import Product
class MockedProduct(Product):
def method_B(self):
self.random_attr = 2
with mock.patch('module.Product', new=MockedProduct):
#Write test case logic here
#Now method_B function call on product class instance should return 2
#instead of 1
minimal reproducible example using pytest and monkeypatch
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test_ab.py
import pytest
from a import A
from b import B
def new_foo(self):
return "New foo"
def test_mock_instance_method(monkeypatch):
y = B()
print(y.bar())
monkeypatch.setattr(A, 'foo', new_foo)
print(y.bar())
gives you
$ pytest -rP .
============================= test session starts ==============================
platform linux -- Python 3.8.8, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /home/user/220225so
plugins: cov-3.0.0
collected 1 item
test_ab.py . [100%]
==================================== PASSES ====================================
__________________________ test_mock_instance_method ___________________________
----------------------------- Captured stdout call -----------------------------
A's foo
New foo
============================== 1 passed in 0.01s ===============================
$