How can I log/print a custom error message when response.raise_for_status() == 404?

Question

I had an instance where the requests library was not catching the 404 and instead passing the code onto another error. To catch a 404, I’ve brought in the raise_for_status() method, which is correctly stopping the code at the HTTPError now. However, it no longer prints my custom error messaging and instead just gives me the stack trace.

Here’s the code

        try:
            response = requests.request(
                request_type, url, headers=headers, data=payload
            )
            response.raise_for_status()
        except requests.ConnectionError as ce:
            logger.info(
                f"""It appears you may need to check
                your internet connection - {ce}"""
            )
        except requests.exceptions.HTTPError as he:

            if response.raise_for_status() == 404:
                logger.info(
                    f"""We seem to be having an issue
                    with your request - {he.response}"""
                )
                logger.info(f"request - {he.request}")
        return response

and here is the full stack trace I’m getting when a 404 is received:

Traceback (most recent call last):
  File "/home/user/projects/project_code/url_help/url_help.py", line 51, in connect_to_url
    response.raise_for_status()
  File "/home/user/projects/project_code/env/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api-url.com/item/v3/companies/8951507/persons?page=1&pageSize=10

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/projects/project_code/main.py", line 99, in <module>
    main(args)
  File "/home/user/projects/project_code/main.py", line 50, in main
    ).get_employees_from_company_id(PAGE_SIZE, COMPANY_ID)
  File "/home/user/projects/project_code/employee_dict/employee_dict.py", line 109, in get_employees_from_company_id
    response = Url.connect_to_url(
  File "/home/user/projects/project_code/url_help/url_help.py", line 60, in connect_to_url
    if response.raise_for_status() == 404:
  File "/home/user/projects/project_code/env/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api-url.com/item/v3/companies/8951507/persons?page=1&pageSize=10

Does anyone have suggestions on how to get this to print/log the message? In all other instances, I can get the custom message to log in addition to the actual error coding.

Asked By: Ryan

||

Source

Answer 1

raise_for_status is a method which raises Error when the http requests fails and returns nothing otherwise.

If you want to check the response status, you should check the status_code, i.e.,

if response.status_code == 404:
    # do sth.

Answered By: BookSword

Answer 2

raise_for_status is not a function that returns the code, but rather a function that will raise an exception if the status code is not ok. Use the status_code attribute instead. For example:

    try:
        response = requests.request(request_type, url, headers=headers, data=payload)

        if response.status_code == 404:
            logger.info(f"""We seem to be having an issue with your request - {he.response}""")
        else:
            response.raise_for_status()

    except requests.ConnectionError as ce:
        ...

If you’d like to still raise an exception in case of 404, just remove the else:, and unindent the following call to response.raise_for_status().

Answered By: micromoses

Answer 3

You’re calling raise_for_status again in the except block. This re-raises an exception, and it’s not caught. Inside except block, you need to check status_code:

        try:
            response = requests.request(
                request_type, url, headers=headers, data=payload
            )
            response.raise_for_status()
        except requests.ConnectionError as ce:
            logger.info(
                f"""It appears you may need to check
                your internet connection - {ce}"""
            )
        except requests.exceptions.HTTPError as he:

            if response.status_code == 404:
                logger.info(
                    f"""We seem to be having an issue
                    with your request - {he.response}"""
                )
                logger.info(f"request - {he.request}")
        return response

Answered By: DimaDK

Answer 4

When you call response.raise_for_status() inside the except block for HTTPError, it is raising another instance of HTTPError and the custom message you are trying to log is not being executed.

To log the custom message, you can access the response object’s text attribute which will contain the error message, and log that instead. Here’s how you can modify the except block for HTTPError:

except requests.exceptions.HTTPError as he:
    if response.status_code == 404:
        logger.info(
            f"We seem to be having an issue with your request - {response.text}"
        )
        logger.info(f"Request - {response.request}")
    else:
        logger.info(f"HTTP error occurred: {he}")

This will log the error message contained in the response.text attribute along with the custom message. If the response status code is not 404, it will log the original HTTPError exception as before.

Note that you don’t need to call response.raise_for_status() again inside the except block. The raise_for_status() method was already called in the try block, and if it did not raise an exception, it means that the response status code is not an error and you don’t need to check it again.

Answered By: Mats Nilsen

How can I log/print a custom error message when response.raise_for_status() == 404?

Question:

Answers: