Python – Mock requests with side_effect

Question:

I’m trying to mock out the method requests.get() of the module requests and set the attribute side_effect of the obtained instance of the Mock class.
I would like to associate a different status_code for each side effect value but I didn’t succeed so far.

def test_func1(mocker):
    side_effect = ["Ok",'','','Failed']
    
    # This line should be changed
    fake_resp.status_code = 200
    
    fake_resp = mocker.Mock()
    fake_resp.json = mocker.Mock(side_effect=side_effect)       
    
    mocker.patch("app.main.requests.get", return_value=fake_resp)
        
    # func1 executes multiple API calls using requests.get()
    # and status_code is needed
    a = func1(a, b) 
    
    assert a == "something"

I have not been able to find a way (in the doc and SO) to associate the status_code for each mock request.

I was thinking about something like this but it’s obviously not working:

def test_func1(mocker):
    side_effect = [(status_code=200, return="Ok"),
                   (status_code=204, return=""), 
                   (status_code=204, return=""),
                   (status_code=500, return="Failed")]
    ....

EDIT: add the code of func1()

from datetime import datetime, timedelta
import requests

def func1(days, delta_1):
    """
    days: number of days before first search (80, 25, 3)
    delta_1: number of days for the date range search (40, 20, 15)
    """
    now = datetime.now()
    start_date = now + timedelta(days=days)
    
    # Var to stop loop when price is found
    loop_stop = 0
    
    # Var to stop loop when search date is more than a year later
    delta_time = 0

    price = 0
    departureDate = "n/a"

    # For loop to check prices till one year. 
    while loop_stop == 0 and delta_time < (365 - days):
        date_range = (
            (start_date + timedelta(days=delta_time)).strftime("%Y%m%d")
            + "-"
            + (start_date + timedelta(days=delta_time + (delta_1 / 2))).strftime(
                "%Y%m%d"
            )
        )
        
        # Needs to be mocked
        response = requests.get("fake_url_using_date_range_var")
        
        if response.status_code == 204:
            print("No data found on this data range")
            delta_time += delta_1
        elif response.status_code == 200:
            price = response.json()["XXX"][0]
            departureDate = response.json()["YYY"][0]
            loop_stop = 1
        else:
            raise NameError(
                response.status_code,
                "Error occured while querying API",
                response.json(),
            )
        
    return price, departureDate
Asked By: baguette

||
Source

Answers:

Possible solution with module unittest (not pytest)

I have written this answer before @baguette added the code of its function func1(), so I have created a file called my_file_01.py which contains my production function func1():

import requests

def func1():
    response1 = 'empty1'
    response2 = 'empty2'
    r = requests.get('http://www.someurl.com')
    if r.status_code == 200:
        response1 = r.json()
    r = requests.get('http://www.some_other_url.com')
    if r.status_code == 500:
        response2 = r.json()
    return [response1, response2]

As you can see func1() calls requests.get() two times and checks the status code of responses.

I have inserted the test code in a different file with the following content:

import unittest
from unittest import mock
from my_file_01 import func1

def request_resp1(url):
    response_mock = mock.Mock()
    response_mock.status_code = 200
    response_mock.json.return_value = {'key1': 'value1'}
    # the function return an instance of class Mock
    return response_mock

def request_resp2(url):
    response_mock = mock.Mock()
    response_mock.status_code = 500
    response_mock.json.return_value = "Failed"
    # the function return an instance of class Mock
    return response_mock

class TestFunc1(unittest.TestCase):
    @mock.patch("my_file_01.requests")
    def test_func1(self, mock_requests):
        print("test func1()")
        mock_requests.get.side_effect = [request_resp1(""), request_resp2("")]
        [response1, response2] = func1()
        print("response1 = " + str(response1))
        print("response2 = " + str(response2))

if __name__ == "__main__":
    unittest.main()

The test file defines the test class TestFunc1 which contains the test method test_func1().

Furthermore the file defines 2 functions called request_resp1() and request_resp2().
These functions are used to define different response values when it is called the method requests.get() by the code of func1(); to be more accurate:

  • the first call to requests.get() assigns to variable r the Mock object with status_code = 200 so it simulates a success;
  • the second call to requests.get() assigns to variable r the Mock object with status_code = 500 so it simulates a failure.

If you try to execute the test code you can see that func1() return different values for the 2 different status_codes of the response. The output of the execution is composed by the 3 print() instructions present in test_func1() and is the followed:

test func1()
response1 = {'key1': 'value1'}
response2 = Failed

Useful links

  • For detail about side_effect attribute see its documentation.

  • This is an other example with side_effect

  • request is a Python module and this is a minimal documentation about it.

Answered By: frankfalse

Based on the solution from @frankfalse, the two mocking functions can be replaced by a class.

class MockResponse:
    def __init__(self, json_data, status_code=requests.codes.ok):
        self.json_data = json_data
        self.status_code = status_code

    def json(self):
        return self.json_data

With the previous class the file contained the test code of @frankfalse becomes:

import unittest
from unittest import mock
from my_file_01 import func1
import requests

class MockResponse:
    def __init__(self, json_data, status_code=requests.codes.ok):
        self.json_data = json_data
        self.status_code = status_code

    def json(self):
        return self.json_data

class TestFunc1(unittest.TestCase):
    @mock.patch("my_file_01.requests")
    def test_func1(self, mock_requests):
        print("test func1()")
        mock_requests.get.side_effect = [MockResponse({'key1': 'value1'}), MockResponse('Failed', 500)]
        [response1, response2] = func1()
        print("response1 = " + str(response1))
        print("response2 = " + str(response2))

The differences are:

  • the mocking functions request_resp1() and request_resp2() can be removed
  • it is necessary to add: import requests for the presence of the assignment status_code=requests.codes.ok in the __init__() method
  • the instruction for the definition of side_effect becomes:
    mock_requests.get.side_effect = [MockResponse({'key1': 'value1'}), MockResponse('error', 500)]

Because ith has been created the class MockResponse, the class Mock of the module unittest.mock is not used.

Answered By: baguette