How to measure redirection latency using Python requests library?

Question

Redirect latency: This time is between 301 and 200 or maybe the time between two redirects is 301.

Like a Chrome extension:

How to measure redirection latency using the requests library?

import requests
r = 'http://httpbin.org/redirect/3'
r = requests.head(r, allow_redirects=True, stream=True)
r.elapsed

r.elapsed is not what I need, it shows the time between sending the request and the first content it receives.

Asked By: Sardar

||

Source

Answer 1

Option 1

Using the requests library, you would have to set the allow_redirects parameter to False and have a loop where you look up for the Location response header (which indicates the URL to redirect a page to), then perform a new request to that URL, and finally, measure the total elapsed time.

However, you may find it easier to do that with httpx library, which is very similar to Python requests, but with more features. You can set the follow_redirects parameter to False (which is the default value anyway) and use the .next_request attribue of the Response object to get the redirection URL into an already constructed Request object. As described earlier, you can then have a loop to send each request and measure their response time (= transport latency + processing time), separately. The response.elapsed returns a timedelta object with the time elapsed from sending the request to the arrival of the response. By adding all the response times together, you can measure the total time elapsed. Example:

import httpx

url = 'http://httpbin.org/redirect/3'

with httpx.Client() as client:
    r = client.get(url, follow_redirects=False)
    print(r.url, r.elapsed, '', sep='n')
    total = r.elapsed.total_seconds()
    
    while 300 < r.status_code < 400:
        r = client.send(r.next_request)
        print(r.url, r.elapsed, '', sep='n')
        total += r.elapsed.total_seconds()
        
    print(f'Total time elapsed: {total} s')

Option 2

Set the follow_redirects parameter to True and use the .history attribute of the Response object to get a list of responses that led to the final URL. The .history property contains a list of any redirect responses that were followed, in the order in which they were made. You can measure the elapsed time for each request, as well as the total elapsed time, as demonstrated in Option 1 above. Example:

import httpx

url = 'http://httpbin.org/redirect/3'

with httpx.Client() as client:
    r = client.get(url, follow_redirects=True)
    
    total = 0
    if r.history:
        for resp in r.history:
            print(resp.url, resp.elapsed, '', sep='n')
            total += resp.elapsed.total_seconds()

    print(r.url, r.elapsed, '', sep='n')
    total += r.elapsed.total_seconds()
    print(f'Total time elapsed: {total} s')

In Python requests (instead of httpx), the above approach would be as follows:

import requests

url = 'http://httpbin.org/redirect/3'

with requests.Session() as session:
    r = session.get(url, allow_redirects=True)
     
    total = 0
    if r.history:
        for resp in r.history:
            print(resp.url, resp.elapsed, '', sep='n')
            total += resp.elapsed.total_seconds()

    print(r.url, r.elapsed, '', sep='n')
    total += r.elapsed.total_seconds()
    print(f'Total time elapsed: {total} s')

Note

If you would like to ignore the body from the response (as you might not need it, and hence, you wouldn’t want to have it loaded into your RAM, especially when using Option 2, where every response is saved to history), you can use HEAD instead of GET request. Examples are given below.

Using httpx:

r = client.head(url, ...

Using requests:

r = session.head(url, ...

Answered By: Chris

How to measure redirection latency using Python requests library?

Question:

Answers:

Option 1

Option 2

Note