Python urllib2: Reading content body even during HTTPError exception?
Question:
I’m using urllib2 to fetch a a page via HTTP. Sometimes the resource throws a HTTP error 400 (Bad Request) when my request contains an error. However, that response also contains an XML element that gives a detailed error message. It would be very handy to be able to see that error rather than just the HTTPError exception returned by urllib2.
How do I return the document contents in spite of the exception?
Answers:
You can treat the error as a response.
http://www.voidspace.org.uk/python/articles/urllib2.shtml#httperror
When an error is raised the server
responds by returning an HTTP error
code and an error page. You can use
the HTTPError instance as a response
on the page returned. This means that
as well as the code attribute, it also
has read, geturl, and info, methods.
import urllib2
try:
request = urllib2.Request('http://www.somesite.com')
response = urllib2.urlopen(req)
except urllib2.HTTPError as e:
error_message = e.read()
print error_message
You can read
the response message from the HTTPError exception.
Python3 example
import urllib.request
try:
request = urllib.request.Request('http://httpstat.us/418', headers={'Accept': 'text/plain', 'User-Agent': ''})
with urllib.request.urlopen(request) as page:
print('success: ' + page.read().decode())
except urllib.error.HTTPError as httpError:
error = httpError.read().decode()
print('error: ' + error)
I’m using urllib2 to fetch a a page via HTTP. Sometimes the resource throws a HTTP error 400 (Bad Request) when my request contains an error. However, that response also contains an XML element that gives a detailed error message. It would be very handy to be able to see that error rather than just the HTTPError exception returned by urllib2.
How do I return the document contents in spite of the exception?
You can treat the error as a response.
http://www.voidspace.org.uk/python/articles/urllib2.shtml#httperror
When an error is raised the server
responds by returning an HTTP error
code and an error page. You can use
the HTTPError instance as a response
on the page returned. This means that
as well as the code attribute, it also
has read, geturl, and info, methods.
import urllib2
try:
request = urllib2.Request('http://www.somesite.com')
response = urllib2.urlopen(req)
except urllib2.HTTPError as e:
error_message = e.read()
print error_message
You can read
the response message from the HTTPError exception.
Python3 example
import urllib.request
try:
request = urllib.request.Request('http://httpstat.us/418', headers={'Accept': 'text/plain', 'User-Agent': ''})
with urllib.request.urlopen(request) as page:
print('success: ' + page.read().decode())
except urllib.error.HTTPError as httpError:
error = httpError.read().decode()
print('error: ' + error)