Changing the referer URL in python requests

Question

How do I change the referer if I’m using the requests library to make a GET request to a web page. I went through the entire manual but couldn’t find it.

Asked By: Mayank Kumar

||

Source

Answer 1

According to http://docs.python-requests.org/en/latest/user/advanced/#session-objects , you should be able to do:

s = requests.Session()
s.headers.update({'referer': my_referer})
s.get(url)

Or just:

requests.get(url, headers={'referer': my_referer})

Your headers dict will be merged with the default/session headers. From the docs:

Any dictionaries that you pass to a request method will be merged with
the session-level values that are set. The method-level parameters
override session parameters.

Answered By: simon

Answer 2

here we are rotating the user_agent with referer

 user_agent_list = [
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36",
            "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36",
            "Mozilla/5.0 (iPad; CPU OS 15_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/104.0.5112.99 Mobile/15E148 Safari/604.1"
        ]
        reffer_list=[
            'https://stackoverflow.com/',
            'https://twitter.com/',
            'https://www.google.co.in/',
            'https://gem.gov.in/'
        ]
        headers = {'Connection': 'keep-alive',
                   'Cache-Control': 'max-age=0',
                   'Upgrade-Insecure-Requests': '1',
                   'User-Agent': random.choice(user_agent_list),
                   'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
                   'Accept-Encoding': 'gzip, deflate',
                   'Accept-Language': 'en-US,en;q=0.9,fr;q=0.8',
                   'referer': random.choice(reffer_list)}

Answered By: DEEPAK JAIN

Changing the referer URL in python requests

Question:

Answers: