Requests — how to tell if you're getting a 404

Question

I’m using the Requests library and accessing a website to gather data from it with the following code:

r = requests.get(url)

I want to add error testing for when an improper URL is entered and a 404 error is returned. If I intentionally enter an invalid URL, when I do this:

print r

I get this:

<Response [404]>

EDIT:

I want to know how to test for that. The object type is still the same. When I do r.content or r.text, I simply get the HTML of a custom 404 page.

Asked By: user1427661

||

Source

Answer 1

Look at the r.status_code attribute:

if r.status_code == 404:
    # A 404 was issued.

Demo:

>>> import requests
>>> r = requests.get('http://httpbin.org/status/404')
>>> r.status_code
404

If you want requests to raise an exception for error codes (4xx or 5xx), call r.raise_for_status():

>>> r = requests.get('http://httpbin.org/status/404')
>>> r.raise_for_status()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "requests/models.py", line 664, in raise_for_status
    raise http_error
requests.exceptions.HTTPError: 404 Client Error: NOT FOUND
>>> r = requests.get('http://httpbin.org/status/200')
>>> r.raise_for_status()
>>> # no exception raised.

You can also test the response object in a boolean context; if the status code is not an error code (4xx or 5xx), it is considered ‘true’:

if r:
    # successful response

If you want to be more explicit, use if r.ok:.

Answered By: Martijn Pieters

Answer 2

If your request is made inside another function, but you want to catch the error in a higher level, it is good to know that you can also get the status code directly from the exception. In my case I could not access the response since the HTTPError was raised before my function was able to pass on the response. I ended up doing the following:

try:
     r = function_calling_request(the_request)
except HTTPError as e:
     if e.response.status_code == 404:
          return do_stuff_if_not_found()

Answered By: Boketto

Requests — how to tell if you're getting a 404

Question:

Answers: