globals and locals in python exec()
Question:
I’m trying to run a piece of python code using exec.
my_code = """
class A(object):
pass
print 'locals: %s' % locals()
print 'A: %s' % A
class B(object):
a_ref = A
"""
global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)
print local_env
which results in the following output
locals: {'A': <class 'A'>}
A: <class 'A'>
Traceback (most recent call last):
File "python_test.py", line 16, in <module>
exec(my_code_AST, global_env, local_env)
File "My Code", line 8, in <module>
File "My Code", line 9, in B
NameError: name 'A' is not defined
However, if I change the code to this –
my_code = """
class A(object):
pass
print 'locals: %s' % locals()
print 'A: %s' % A
class B(A):
pass
"""
global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)
print local_env
then it works fine – giving the following output –
locals: {'A': <class 'A'>}
A: <class 'A'>
{'A': <class 'A'>, 'B': <class 'B'>}
Clearly A is present and accessible – what’s going wrong in the first piece of code? I’m using 2.6.5, cheers,
Colin
* UPDATE 1 *
If I check the locals() inside the class –
my_code = """
class A(object):
pass
print 'locals: %s' % locals()
print 'A: %s' % A
class B(object):
print locals()
a_ref = A
"""
global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)
print local_env
Then it becomes clear that locals() is not the same in both places –
locals: {'A': <class 'A'>}
A: <class 'A'>
{'__module__': '__builtin__'}
Traceback (most recent call last):
File "python_test.py", line 16, in <module>
exec(my_code_AST, global_env, local_env)
File "My Code", line 8, in <module>
File "My Code", line 10, in B
NameError: name 'A' is not defined
However, if I do this, there is no problem –
def f():
class A(object):
pass
class B(object):
a_ref = A
f()
print 'Finished OK'
* UPDATE 2 *
ok, so the docs here – http://docs.python.org/reference/executionmodel.html
‘A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution. The namespace of the class definition becomes the attribute dictionary of the class. Names defined at the class scope are not visible in methods.’
It seems to me that ‘A’ should be made available as a free variable within the executable statement that is the definition of B, and this happens when we call f() above, but not when we use exec(). This can be more easily shown with the following –
my_code = """
class A(object):
pass
print 'locals in body: %s' % locals()
print 'A: %s' % A
def f():
print 'A in f: %s' % A
f()
class B(object):
a_ref = A
"""
which outputs
locals in body: {'A': <class 'A'>}
A: <class 'A'>
Traceback (most recent call last):
File "python_test.py", line 20, in <module>
exec(my_code_AST, global_env, local_env)
File "My Code", line 11, in <module>
File "My Code", line 9, in f
NameError: global name 'A' is not defined
So I guess the new question is – why aren’t those locals being exposed as free variables in functions and class definitions – it seems like a pretty standard closure scenario.
Answers:
Well, I believe it’s either an implementation bug or an undocumented design decision. The crux of the issue is that a name-binding operation in the module-scope should bind to a global variable. The way it is achieved is that when in the module level, globals() IS locals() (try that one out in the interpreter), so when you do any name-binding, it assigns it, as usual, to the locals() dictionary, which is also the globals, hence a global variable is created.
When you look up a variable, you first check your current locals, and if the name is not found, you recursively check locals of containing scopes for the variable name until you find the variable or reach the module-scope. If you reach that, you check the globals, which are supposed to be the module scope’s locals.
>>> exec(compile("import sysnprint sys._getframe().f_code.co_name", "blah", "exec"), {}, {})
<module>
>>> exec("a = 1nclass A(object):ntprint an", {}, {})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 2, in <module>
File "<string>", line 3, in A
NameError: name 'a' is not defined
>>> d = {}
>>> exec("a = 1nclass A(object):ntprint an", d,d)
1
This behavior is why inheritance worked (The name-lookup used code object’s scope locals(), which indeed had A in it).
In the end, it’s an ugly hack in the CPython implementation, special-casing globals lookup. It also causes some nonsensical artifical situations – e.g.:
>>> def f():
... global a
... a = 1
...
>>> f()
>>> 'a' in locals()
True
Please note that this is all my inference based on messing with the interpreter while reading section 4.1 (Naming and binding) of the python language reference. While this isn’t definitive (I haven’t opened CPython’s sources), I’m fairly sure I’m correct about the behavior.
If your question is how to get the exec statement to behave like the file scope, I followed some hints in the linked question and bug and got it working by passing a single dictionary for globals and locals. Apparently the file scope is a special case where local declarations are automatically placed in the global scope.
exec code in dict()
After print locals() and globals(),you will find the reason why exec throws “not defined” exception, and you can try this
d = dict(locals(), **globals())
exec (code, d, d)
my_code = """
class A(object):
pass
class B(object):
a = A
"""
my_code_AST = compile(my_code, "My Code", "exec")
extra_global = "hi"
global_env = {}
exec my_code_AST in global_env
print "global_env.keys() =", global_env.keys()
print "B.a =", global_env["B"].a
prints
global_env.keys() = ['__builtins__', 'A', 'B']
B.a = <class 'A'>
Hawkett, you say,
the main reason I wanted to use locals like that, was to get all the stuff defined in the code string, without all the other stuff that python puts in the globals.
With exec, if your globals don’t have __builtins__ defined, exec adds one item, __builtins__ to your globals, so you get A, B, and __builtins__. __builtins__ itself is a big dictionary, but it’s always the same one element to delete (as long as you wait until your code is finished using it before you delete it!). Documented under exec() here.
The docs for eval under built in functions say
If the globals dictionary is present and lacks ‘builtins’, the current globals are copied into globals before expression is parsed.
But actually it seems only to copy __builtins__ in.
(And n.b. what everyone else said: either set globals and locals the same or say exec my_code_AST in global_env without a separate local_env.)
I’m trying to run a piece of python code using exec.
my_code = """
class A(object):
pass
print 'locals: %s' % locals()
print 'A: %s' % A
class B(object):
a_ref = A
"""
global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)
print local_env
which results in the following output
locals: {'A': <class 'A'>}
A: <class 'A'>
Traceback (most recent call last):
File "python_test.py", line 16, in <module>
exec(my_code_AST, global_env, local_env)
File "My Code", line 8, in <module>
File "My Code", line 9, in B
NameError: name 'A' is not defined
However, if I change the code to this –
my_code = """
class A(object):
pass
print 'locals: %s' % locals()
print 'A: %s' % A
class B(A):
pass
"""
global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)
print local_env
then it works fine – giving the following output –
locals: {'A': <class 'A'>}
A: <class 'A'>
{'A': <class 'A'>, 'B': <class 'B'>}
Clearly A is present and accessible – what’s going wrong in the first piece of code? I’m using 2.6.5, cheers,
Colin
* UPDATE 1 *
If I check the locals() inside the class –
my_code = """
class A(object):
pass
print 'locals: %s' % locals()
print 'A: %s' % A
class B(object):
print locals()
a_ref = A
"""
global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)
print local_env
Then it becomes clear that locals() is not the same in both places –
locals: {'A': <class 'A'>}
A: <class 'A'>
{'__module__': '__builtin__'}
Traceback (most recent call last):
File "python_test.py", line 16, in <module>
exec(my_code_AST, global_env, local_env)
File "My Code", line 8, in <module>
File "My Code", line 10, in B
NameError: name 'A' is not defined
However, if I do this, there is no problem –
def f():
class A(object):
pass
class B(object):
a_ref = A
f()
print 'Finished OK'
* UPDATE 2 *
ok, so the docs here – http://docs.python.org/reference/executionmodel.html
‘A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution. The namespace of the class definition becomes the attribute dictionary of the class. Names defined at the class scope are not visible in methods.’
It seems to me that ‘A’ should be made available as a free variable within the executable statement that is the definition of B, and this happens when we call f() above, but not when we use exec(). This can be more easily shown with the following –
my_code = """
class A(object):
pass
print 'locals in body: %s' % locals()
print 'A: %s' % A
def f():
print 'A in f: %s' % A
f()
class B(object):
a_ref = A
"""
which outputs
locals in body: {'A': <class 'A'>}
A: <class 'A'>
Traceback (most recent call last):
File "python_test.py", line 20, in <module>
exec(my_code_AST, global_env, local_env)
File "My Code", line 11, in <module>
File "My Code", line 9, in f
NameError: global name 'A' is not defined
So I guess the new question is – why aren’t those locals being exposed as free variables in functions and class definitions – it seems like a pretty standard closure scenario.
Well, I believe it’s either an implementation bug or an undocumented design decision. The crux of the issue is that a name-binding operation in the module-scope should bind to a global variable. The way it is achieved is that when in the module level, globals() IS locals() (try that one out in the interpreter), so when you do any name-binding, it assigns it, as usual, to the locals() dictionary, which is also the globals, hence a global variable is created.
When you look up a variable, you first check your current locals, and if the name is not found, you recursively check locals of containing scopes for the variable name until you find the variable or reach the module-scope. If you reach that, you check the globals, which are supposed to be the module scope’s locals.
>>> exec(compile("import sysnprint sys._getframe().f_code.co_name", "blah", "exec"), {}, {})
<module>
>>> exec("a = 1nclass A(object):ntprint an", {}, {})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 2, in <module>
File "<string>", line 3, in A
NameError: name 'a' is not defined
>>> d = {}
>>> exec("a = 1nclass A(object):ntprint an", d,d)
1
This behavior is why inheritance worked (The name-lookup used code object’s scope locals(), which indeed had A in it).
In the end, it’s an ugly hack in the CPython implementation, special-casing globals lookup. It also causes some nonsensical artifical situations – e.g.:
>>> def f():
... global a
... a = 1
...
>>> f()
>>> 'a' in locals()
True
Please note that this is all my inference based on messing with the interpreter while reading section 4.1 (Naming and binding) of the python language reference. While this isn’t definitive (I haven’t opened CPython’s sources), I’m fairly sure I’m correct about the behavior.
If your question is how to get the exec statement to behave like the file scope, I followed some hints in the linked question and bug and got it working by passing a single dictionary for globals and locals. Apparently the file scope is a special case where local declarations are automatically placed in the global scope.
exec code in dict()
After print locals() and globals(),you will find the reason why exec throws “not defined” exception, and you can try this
d = dict(locals(), **globals())
exec (code, d, d)
my_code = """
class A(object):
pass
class B(object):
a = A
"""
my_code_AST = compile(my_code, "My Code", "exec")
extra_global = "hi"
global_env = {}
exec my_code_AST in global_env
print "global_env.keys() =", global_env.keys()
print "B.a =", global_env["B"].a
prints
global_env.keys() = ['__builtins__', 'A', 'B']
B.a = <class 'A'>
Hawkett, you say,
the main reason I wanted to use locals like that, was to get all the stuff defined in the code string, without all the other stuff that python puts in the globals.
With exec, if your globals don’t have __builtins__ defined, exec adds one item, __builtins__ to your globals, so you get A, B, and __builtins__. __builtins__ itself is a big dictionary, but it’s always the same one element to delete (as long as you wait until your code is finished using it before you delete it!). Documented under exec() here.
The docs for eval under built in functions say
If the globals dictionary is present and lacks ‘builtins’, the current globals are copied into globals before expression is parsed.
But actually it seems only to copy __builtins__ in.
(And n.b. what everyone else said: either set globals and locals the same or say exec my_code_AST in global_env without a separate local_env.)