Save a large file using the Python requests library

Question

I know that fetching a url is as simple as requests.get and I can get at the raw response body and save it to a file, but for large files, is there a way to stream directly to a file? Like if I’m downloading a movie with it or something?

Asked By: Matt Williamson

||

Source

Answer 1

Oddly enough, requests doesn’t have anything simple for this. You’ll have to iterate over the response and write those chunks to a file:

response = requests.get('http://www.example.com/image.jpg', stream=True)

# Throw an error for bad status codes
response.raise_for_status()

with open('output.jpg', 'wb') as handle:
    for block in response.iter_content(1024):
        handle.write(block)

I usually just use urllib.urlretrieve(). It works, but if you need to use a session or some sort of authentication, the above code works as well.

Answered By: Blender

Save a large file using the Python requests library

Question:

Answers: