How can I set a single proxy for a requests session object?

Question

I’m using the Python requests package to send http requests. I want to add a single proxy to the requests session object. eg.

session = requests.Session()
session.proxies = {...} # Here I want to add a single proxy

Currently I am looping through a bunch of proxies, and at each iteration a new session is made. I only want to set a single proxy for each iteration.

The only example I see in the documentation is:

proxies = {
    "http": "http://10.10.1.10:3128",
    "https": "http://10.10.1.10:1080",
}

requests.get("http://example.org", proxies=proxies)

I’ve tried to follow this, but to no avail. Here is my code from the script:

# eg. line = 59.43.102.33:80
r = s.get('http://icanhazip.com', proxies={'http': 'http://' + line})

But I get an error:

requests.packages.urllib3.exceptions.LocationParseError: Failed to parse 59.43.102.33:80

How is it possible to set a single proxy on a session object?

Asked By: Torra

||

Source

Answer 1

In fact, you are right, but you must ensure your defination of ‘line’, I have tried this , it’s ok:

>>> import requests
>>> s = requests.Session()
>>> s.get("http://www.baidu.com", proxies={'http': 'http://10.11.4.254:3128'})
<Response [200]>

Did you define the line like line = ' 59.43.102.33:80', there is a space at the front of address.

Answered By: NeoWu

Answer 2

In addition to @neowu’ answer, if you would like to set a proxy for the lifetime of a session object, you can also do the following –

import requests
proxies = {'http': 'http://10.11.4.254:3128'}
s = requests.session()
s.proxies.update(proxies)
s.get("http://www.example.com")   # Here the proxies will also be automatically used because we have attached those to the session object, so no need to pass separately in each call

Answered By: Vikas Ojha

Answer 3

Hopefully this may lead to an answer:

urllib3.util.url.parse_url(url)
Given a url, return a parsed Url namedtuple. Best-effort is performed to parse incomplete urls. Fields not provided will be None.

retrived from https://urllib3.readthedocs.org/en/latest/helpers.html

Answered By: user4642224

Answer 4

There are other ways you can set proxies, apart from the solutions you have got so far:

import requests

with requests.Session() as s:
    # either like this
    s.proxies = {'https': 'http://105.234.154.195:8888', 'http': 'http://199.188.92.69:8000'}
    # or like this
    s.proxies['https'] = 'http://105.234.154.195:8888'
    r = s.get(link)

Answered By: SIM

How can I set a single proxy for a requests session object?

Question:

Answers: