Changing monkeypatch setattr multiple times

Question

I am trying to test code that depends on a third party and would like to use monkeypatch to replicate what I expect a request will return. Here is a minimal example of the code that I have.

import requests

def get_urls(*urls):
    results = []
    for url in urls:
        results.append(requests.get(url).text)

For my tests, I have something like the following:

from my_package import get_urls

def test_get_urls():
    urls = ("https://example.com/a", "https://example.com/b", "https://example.com/c")
    assert len(get_urls(urls)) == 3

How can I monkeypatch each of the calls to requests.get using monkeypatch.setattr? The mock package seems to be able to do this using side effects. How do I do this with pytest?

Asked By: Jack Moody

||

Source

Answer 1

When you override a method call using monkeypatch, you can set that attribute to be a custom function. Here is one method of implementing different behaviors based on url:

URL_MAP = {
    'https://example.com/a': json.dumps({1: 2}),
    'https://example.com/b': json.dumps({3: 4})
}

def fake_req_get(url, *args, **kwargs):
    return URL_MAP.get(url, '{}')

def test_get_urls(monkeypatch):
    monkeypatch.setattr('requests.get', fake_req_get)
    urls = ("https://example.com/a", "https://example.com/b", "https://example.com/c")
    assert get_urls(urls)[0] == URL_MAP["https://example.com/a"]

Answered By: jordanm

Answer 2

I had a similar situation (but not an identical one), which the accepted answer helped me with, but there was a little extra thinking I had to do to finish figuring out how to do this for my particular use case. It took me long enough that I thought I would share the fruits of my labor here since it led to a more generalizable way of doing things.

If you’re doing a test where the function you’re executing, (in this example get_urls), calls the monkeypatched thing (here requests.get) more than one time, and you want the result of the monkeypatched thing to be a certain value the first time, and another value the second time, you can create a class with stateful info involving what you want the first, second, … calls to it to return, plus an index for which call you’re on, with a __call__ attribute so that an instance of the class still seems like a normal function to the caller. This makes the return value of the thing you’re monkeypatching similar to the progression of values as you go through an iterator. For example:

def test_mock_a_function(monkeypatch):
    class fake_random_twice:
        vals = ['a', 'c']
        val_idx = 0
        def __call__(self):
            val = self.vals[self.val_idx]
            self.val_idx += 1
            return val

    def to_exercise():
        import random
        val1 = random.random()
        val2 = 'b'
        val3 = random.random()
        return val1 + val2 + val3

    # Remember, `fake_random_twice()` *instantiates* a callable, but doesn't call it yet,
    # so `to_exercise`'s calls to random.random() will return actual values.
    monkeypatch.setattr('random.random', fake_random_twice())
    assert to_exercise() == 'abc'

Test passes! 🙂

Or, in the case where you’re already mocking a method of a class (which was my exact usecase, here for mocking datetime.now), you do something like this:

def test_mock_a_method(monkeypatch):
    mocked_start_time = datetime.datetime.now()
    # Pretend we next call now() 100 milliseconds later.
    mocked_end_time = mocked_start_time + datetime.timedelta(microseconds=100)

    class datetime_mocker:
        # Return first the mocked start time and second the mocked end time. This works because
        # datetime.datetime.now() is called twice by the thing we're testing; if it instead called
        # datetime.datetime.now() eg. three times, we would want to list out three separate times here.
        timestamps_to_return = [mocked_start_time, mocked_end_time]
        # Stateful variable used by now() to accomplish our goal of returning first one time, then
        # the next.
        timestamp_idx = 0

        @classmethod
        def now(cls, timezone):
            this_calls_timestamp = cls.timestamps_to_return[cls.timestamp_idx]
            cls.timestamp_idx += 1
            return this_calls_timestamp

    def to_exercise():
        import datetime

        start_time = datetime.datetime.now(datetime.timezone.utc)
        # ... Do stuff here that takes time which we care about
        # We can tell the mock is working since we'll assert the elapsed value is only 100 ms - but
        # time.sleep(1) ensures that if we were using the real-life datetime, the elapsed value
        # would be >= 1 second.
        time.sleep(1)
        end_time = datetime.datetime.now(datetime.timezone.utc)
        elapsed = end_time - start_time
        return elapsed

    monkeypatch.setattr(datetime, 'datetime', datetime_mocker)
    assert to_exercise() == datetime.timedelta(microseconds=100)

Test also passes! 🙂

Hopes this helps someone else one day.

Answered By: sinback

Changing monkeypatch setattr multiple times

Question:

Answers: