Opening Local File Works with urllib but not with urllib2 Question: I’m trying to open a local file using urllib2. How can I go about doing this? When I try the following line with urllib: resp = urllib.urlopen(url) it works correctly, but when I switch it to: resp = urllib2.urlopen(url) I get: ValueError: unknown url …
How to correctly parse UTF-8 encoded HTML to Unicode strings with BeautifulSoup? Question: I’m running a Python program which fetches a UTF-8-encoded web page, and I extract some text from the HTML using BeautifulSoup. However, when I write this text to a file (or print it on the console), it gets written in an unexpected …
Python Requests: Post JSON and file in single request Question: I need to do a API call to upload a file along with a JSON string with details about the file. I am trying to use the python requests lib to do this: import requests info = { ‘var1’ : ‘this’, ‘var2’ : ‘that’, } …
python ignore certificate validation urllib2 Question: I want to ignore the certification validation during my request to the server with an internal corporate link. With python requests library I would do this: r = requests.get(link, allow_redirects=False,verify=False) How do I do the same with urllib2 library? Asked By: Sangamesh Hs || Source Answers: urllib2 does not …
Submitting to a web form using python Question: I have seen questions like this asked many many times but none are helpful Im trying to submit data to a form on the web ive tried requests, and urllib and none have worked for example here is code that should search for the [python] tag on …
UnicodeEncodeError: 'ascii' codec can't encode character u'u2026' Question: I’m learning about urllib2 and Beautiful Soup and on first tests am getting errors like: UnicodeEncodeError: ‘ascii’ codec can’t encode character u’u2026′ in position 10: ordinal not in range(128) There seem to be lots of posts about this type of error and I have tried the solutions …
Python: URLError: <urlopen error [Errno 10060] Question: OS: Windows 7; Python 2.7.3 using the Python GUI Shell I’m trying to read a website through Python, and several authors use the urllib and urllib2 libraries. To store the site in a variable, I’ve seen a similar approach proposed: import urllib import urllib2 g = “http://www.google.com/” read …
Download and decompress gzipped file in memory? Question: I would like to download a file using urllib and decompress the file in memory before saving. This is what I have right now: response = urllib2.urlopen(baseURL + filename) compressedFile = StringIO.StringIO() compressedFile.write(response.read()) decompressedFile = gzip.GzipFile(fileobj=compressedFile, mode=’rb’) outfile = open(outFilePath, ‘w’) outfile.write(decompressedFile.read()) This ends up writing empty …
urllib.quote() throws KeyError Question: To encode the URI, I used urllib.quote(“schönefeld”) but when some non-ascii characters exists in string, it thorws KeyError: u’xe9′ Code: return ”.join(map(quoter, s)) My input strings are köln, brønshøj, schönefeld etc. When I tried just printing statements in windows(Using python2.7, pyscripter IDE). But in linux it raises exception (I guess platform …
A good way to get the charset/encoding of an HTTP response in Python Question: Looking for an easy way to get the charset/encoding information of an HTTP response using Python urllib2, or any other Python library. >>> url = ‘http://some.url.value’ >>> request = urllib2.Request(url) >>> conn = urllib2.urlopen(request) >>> response_encoding = ? I know that …
