Fetching and evaluating a Python code from an HTTP response in Python 3.X

Question

I’m writing a Python script that gets an HTTP request as its input (namely, a url and some GET parameters). The response to the HTTP request is a piece of Python code representing a big Python dictionary. My script should evaluate the retrieved code and process the evaluated dictionary.

Had I been using Python 2.X, I would have done the follwoing:

import urllib
d = eval(urllib.request.urlopen(input_url).read())
process(d)

The problem is that I’m using Python 3.4, and the read method returns a bytes and not a string as in Python 2.X, so I can’t use the eval function on it.

How do I evaluate the retrieved dictionary on Python 3.X?

Asked By: snakile

||

Source

Answer 1

As @rawing said in the comments, you’ll need to decode the bytes into a string. You’ll also probably want to use ast.literal_eval over eval, as the latter executes arbitrary code, and is unsafe.

Here’s an example where I’ve uploaded a dict of literals as a Github gist:

import urllib.request
import ast

url = ("https://gist.githubusercontent.com/eldridgejm/76c78b7d11a66162687b/" +
      "raw/60a76770970715f859d1e3d33c8e2afcac296a31/gistfile1.txt")

r = urllib.request.urlopen(url).read()
d = ast.literal_eval(r.decode())

print(d)

Running the above prints:

{'bar': 42, 'baz': 'apple', 'foo': 41}

Answered By: jme

Answer 2

Using jme’s example and if it is valid json which I think it may well be you can just use requests and use .json:

import requests

r  = requests.get("https://gist.githubusercontent.com/eldridgejm/76c78b7d11a66162687b/raw/60a76770970715f859d1e3d33c8e2afcac296a31/gistfile1.txt")
print(r.json())
{u'baz': u'apple', u'foo': 41, u'bar': 42}

Answered By: Padraic Cunningham

Fetching and evaluating a Python code from an HTTP response in Python 3.X

Question:

Answers: