How to save "complete webpage" not just basic html using Python Question: I am using following code to save webpage using Python: import urllib import sys from bs4 import BeautifulSoup url = ‘http://www.vodafone.de/privat/tarife/red-smartphone-tarife.html’ f = urllib.urlretrieve(url,’test.html’) Problem: This code saves html as basic html without javascripts, images etc. I want to save webpage as complete …
Python POST binary data Question: I am writing some code to interface with redmine and I need to upload some files as part of the process, but I am not sure how to do a POST request from python containing a binary file. I am trying to mimic the commands here: curl –data-binary “@image.png” -H …
Login to website using urllib2 – Python 2.7 Question: Okay, so I am using this for a reddit bot, but I want to be able to figure out HOW to log in to any website. If that makes sense…. I realise that different websites use different login forms etc. So how do I figure out …
Python urllib2: Receive JSON response from url Question: I am trying to GET a URL using Python and the response is JSON. However, when I run import urllib2 response = urllib2.urlopen(‘https://api.instagram.com/v1/tags/pizza/media/XXXXXX’) html=response.read() print html The html is of type str and I am expecting a JSON. Is there any way I can capture the response …
How to download image using requests Question: I’m trying to download and save an image from the web using python’s requests module. Here is the (working) code I used: img = urllib2.urlopen(settings.STATICMAP_URL.format(**data)) with open(path, ‘w’) as f: f.write(img.read()) Here is the new (non-working) code using requests: r = requests.get(settings.STATICMAP_URL.format(**data)) if r.status_code == 200: img = …
How do I open an image from the internet in PIL? Question: I would like to find the dimensions of an image on the internet. I tried using from PIL import Image import urllib2 as urllib fd = urllib.urlopen(“http://a/b/c”) im = Image.open(fd) im.size as suggested in this answer, but I get the error message addinfourl …
Why do I get "'str' object has no attribute 'read'" when trying to use `json.load` on a string? Question: In Python I’m getting an error: Exception: (<type ‘exceptions.AttributeError’>, AttributeError("’str’ object has no attribute ‘read’",), <traceback object at 0x1543ab8>) Given python code: def getEntries (self, sub): url = ‘http://www.reddit.com/’ if (sub != ”): url += ‘r/’ …
Python-Requests close http connection Question: I was wondering, how do you close a connection with Requests (python-requests.org)? With httplib it’s HTTPConnection.close(), but how do I do the same with Requests? Code: r = requests.post("https://stream.twitter.com/1/statuses/filter.json", data={‘track’:toTrack}, auth=(‘username’, ‘passwd’)) for line in r.iter_lines(): if line: self.mongo[‘db’].tweets.insert(json.loads(line)) Asked By: user179169 || Source Answers: As discussed here, there really …
urllib2 having trouble processing colon symbol in url Question: I am using the API for challonge and their url format is https://username:[email protected]/api/. However, when I use urllib2 in python to get this url response = urllib2.urlopen(‘https://username:[email protected]/api/’) I get an error about a non-numerical port number. I believe this is caused by the colon (:) in …
How to retry urllib2.request when fails? Question: When urllib2.request reaches timeout, a urllib2.URLError exception is raised. What is the pythonic way to retry establishing a connection? Asked By: iTayb || Source Answers: I would use a retry decorator. There are other ones out there, but this one works pretty well. Here’s how you can use …
