How do I clear cache with Python Requests?

Question:

Does the requests package of Python cache data by default?

For example,

import requests
resp = requests.get('https://some website')

Will the response be cached? If so, how do I clear it?

Asked By: ethanjyx

||

Answers:

Python-requests doesn’t have any caching features.

However, if you need them you can look at requests-cache, although I never used it.

Answered By: user2629998

Requests does not do caching by default. You can easily plug it in by using something like CacheControl.

Answered By: Lukasa

Add a 'Cache-Control: no-cache' header:

self.request = requests.get('http://google.com',
                            headers={'Cache-Control': 'no-cache'})

See https://stackoverflow.com/a/55613686/469045 for complete answer.

Answered By: jones77

Late answer, but python requests doesn’t cache requests, you should use the Cache-Control and Pragma headers instead, i.e.:

import requests
h = {
    ...
    "Cache-Control": "no-cache",
    "Pragma": "no-cache"
}
r = requests.get("url", headers=h)
...

HTTP/Headers

  • Cache-Control
    The Cache-Control general-header field is used to specify directives for caching mechanisms in both requests and responses. Caching directives are unidirectional, meaning that a given directive in a request is not implying that the same directive is to be given in the response.

  • Pragma
    Implementation-specific header that may have various effects anywhere
    along the request-response chain. Used for backwards compatibility
    with HTTP/1.0 caches where the Cache-Control header is not yet
    present.


Directive

  • no-cache
    Forces caches to submit the request to the origin server for
    validation before releasing a cached copy.


Note on Pragma:

Pragma is not specified for HTTP responses and is therefore not a
reliable replacement for the general HTTP/1.1 Cache-Control header,
although it does behave the same as Cache-Control: no-cache, if the
Cache-Control header field is omitted in a request. Use Pragma only
for backwards compatibility with HTTP/1.0 clients.

Answered By: Pedro Lobito

I was getting outdated version of a website and I thought about requests cache too, but adding no-cache parameter to headers didn’t change anything. It appears that the cookie I was passing, was causing the server to present outdated site.

Answered By: Human programmer
import requests
h = {
    ...
    "Cache-Control": "no-cache",
    "Pragma": "no-cache",
    "Expires": "0"
}
r = requests.get("url", headers=h)
Answered By: ofir_aghai
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.