When to use `raise_for_status` vs `status_code` testing

Question

I have always used:

r = requests.get(url)
if r.status_code == 200:
    # my passing code
else:
    # anything else, if this even exists

Now I was working on another issue and decided to allow for other errors and am instead now using:

try:
    r = requests.get(url)
    r.raise_for_status()
except requests.exceptions.ConnectionError as err:
    # eg, no internet
    raise SystemExit(err)
except requests.exceptions.HTTPError as err:
    # eg, url, server and other errors
    raise SystemExit(err)
# the rest of my code is going here

With the exception that various other errors could be tested for at this level, is one method any better than the other?

Asked By: Madivad

||

Source

Answer 1

Response.raise_for_status() is just a built-in method for checking status codes and does essentially the same thing as your first example.

There is no "better" here, just about personal preference with flow control. My preference is toward try/except blocks for catching errors in any call, as this informs the future programmer that these conditions are some sort of error. If/else doesn’t necessarily indicate an error when scanning code.

Edit: Here’s my quick-and-dirty pattern.

import time
from http import HTTPStatus

import requests
from requests.exceptions import HTTPError

url = "https://theurl.com"
retries = 3
retry_codes = [
    HTTPStatus.TOO_MANY_REQUESTS,
    HTTPStatus.INTERNAL_SERVER_ERROR,
    HTTPStatus.BAD_GATEWAY,
    HTTPStatus.SERVICE_UNAVAILABLE,
    HTTPStatus.GATEWAY_TIMEOUT,
]

for n in range(retries):
    try:
        response = requests.get(url)
        response.raise_for_status()

        break

    except HTTPError as exc:
        code = exc.response.status_code
        
        if code in retry_codes:
            # retry after n seconds
            time.sleep(n)
            continue

        raise

However, in most scenarios, I subclass requests.Session, make a custom HTTPAdapter that handles exponential backoffs, and the above lives in an overridden requests.Session.request method. An example of that can be seen here.

Answered By: Sam Morgan

Answer 2

Better is somewhat subjective; both can get the job done. That said, as a relatively inexperienced programmer I prefer the Try / Except form.
For me, the T / E reminds me that requests don’t always give you what you expect (in a way that if / else doesn’t – but that could just be me).

raise_for_status() also lets you easily implement as many or as few different actions for the different error types (.HTTPError, .ConnectionError) as you need.
In my current project, I’ve settled on the form below, as I’m taking the same action regardless of cause, but am still interested to know the cause:

    try:
        ...
    except requests.exceptions.RequestException as e:
        raise SystemExit(e) from None

Toy implementation:

import requests

def http_bin_repsonse(status_code):
    sc = status_code
    try:
        url = "http://httpbin.org/status/" + str(sc)
        response = requests.post(url)
        response.raise_for_status()

        p = response.content

    except requests.exceptions.RequestException as e:
        print("placeholder for save file / clean-up")
        raise SystemExit(e) from None
    
    return response, p

response, p = http_bin_repsonse(403)
print(response.status_code)

Answered By: Alcognito

Answer 3

Almost always, raise_for_status() is better.

The main reason is that there is a bit more to it than testing status_code == 200, and you should be making best use of tried-and-tested code rather than creating your own implementation.

For instance, did you know that there are actually five different ‘success’ codes defined by the HTTP standard? Four of those ‘success’ codes will be misinterpreted as failure by testing for status_code == 200.

Answered By: Ian Goldby

Answer 4

If you are not sure, follow the Ian Goldby’s answer.

…however please be aware that raise_for_status() is not some magical or exceptionally smart solution – it’s a very simple function that decodes the response body and throws an exception for HTTP codes 400-599, distinguishing client-side and server-side errors (see its code here).

And especially the client-side error responses may contain valuable information in the response body that you may want to process. For example a HTTP 400 Bad Request response may contain the error reason.

In such a case it may be better to not use raise_for_status() but instead cover all the cases by yourself.

Example code

try:
    r = requests.get(url)

    # process the specific codes from the range 400-599
    # that you are interested in first
    if r.status_code == 400:
        invalid_request_reason = r.text
        print(f"Your request has failed because: {invalid_request_reason}")
        return
    # this will handle all other errors
    elif r.status_code > 400:
        print(f"Your request has failed with status code: {r.status_code}")
        return

except requests.exceptions.ConnectionError as err:
    # eg, no internet
    raise SystemExit(err)

# the rest of my code is going here

Real-world use case

PuppetDB’s API using the Puppet Query Language (PQL) responds with a HTTP 400 Bad Request to a syntactically invalid query with a very precise info where is the error.

Request query:

nodes[certname] { certname == "bastion" }

Body of the HTTP 400 response:

PQL parse error at line 1, column 29:

nodes[certname] { certname == "bastion" }
                            ^

Expected one of:

[
false
true
#"[0-9]+"
-
'
"
#"s+"

See my Pull Request to an app that uses this API to make it show this error message to a user here, but note that it doesn’t exactly follow the example code above.

Answered By: Greg Dubicki

When to use `raise_for_status` vs `status_code` testing

Question:

Answers:

Example code

Real-world use case