strange behaviour of locals() dictionaries when using eval

Question

Please consider this code:

from sklearn.model_selection import ParameterGrid

def conf1():
    return {"p1": [1,2], "p2": [4,5,6]}

def f(aa, bb, cc):
    c = locals()
    print("CONFIG, locals():")
    for k, v, in c.items():
        print(f"{k}: {v}")
    
    #>
    params = eval("conf1()")
    grid = ParameterGrid(params)
    #<

    d = {**c}
    
    print("CONF after eval")
    for k, v, in d.items():
        print(f"{k}: {v}")
    

f(1,2,3)

whose output is:

CONFIG, locals():
aa: 1
bb: 2
cc: 3
CONF after eval
aa: 1
bb: 2
cc: 3
c: {'aa': 1, 'bb': 2, 'cc': 3, 'c': {...}, 'k': 'cc', 'v': 3}
k: cc
v: 3

The function f creates a dictionary c with its local names, specifically the parameter it receives. Then a new dictionary is created by eval("conf1()"). After that, a new dictionary is created d = {**c}.

It seems that when c = {**d} is executed d = locals() is re-evaluated. Since at that point the local environment contains also k, v and d they end up in d. If I remove eval (the lines between #> and #<), I get what I expect, i.e., c and d contains the same items.

May be eval is not the reason, but I cannot really understand what is happening. I cannot find anything explaining this. Any help?

Since in the comments I read that config is a name of a mutable "locals()" I made this test:

def f(param1, param2):
    config = locals()
    print("1. config:")
    for k, v in config.items():
        print(f"- {k}: {v}")
        
    name =1 
    
    print("2. config:")
    for k, v in config.items():
        print(f"- {k}: {v}")
    
f(1, 2)

with output:

1. config:
- param2: 2
- param1: 1
2. config:
- param2: 2
- param1: 1

So it does not seem so mutable.

If one adds an eval:

def f(param1, param2):
    config = locals()
    print("1. config:")
    for k, v in config.items():
        print(f"- {k}: {v}")
        
    name = 1 
    eval("print()")
    
    print("2. config:")
    for k, v in config.items():
        print(f"- {k}: {v}")
    
f(1, 2)

again:

1. config:
- param2: 2
- param1: 1

2. config:
- param2: 2
- param1: 1
- name: 1
- v: 1
- k: param1
- config: {'param2': 2, 'param1': 1, 'name': 1, 'v': 1, 'k': 'param1', 'config': {...}}

After the comment of @juanpa.arrivillaga, I got it and was able to replicate without eval():

def f(param1, param2):
    config = locals()
    print("1. config:")
    for k, v in config.items():
        print(f"- {k}: {v}")
        
    name = "hello"
    locals()
    
    print("2. config:")
    for k, v in config.items():
        print(f"- {k}: {v}")
    
f(1, 2)

with output:

1. config:
- param2: 2
- param1: 1
2. config:
- param2: 2
- param1: 1
- name: hello
- v: 1
- k: param1
- config: {'param2': 2, 'param1': 1, 'name': 'hello', 'v': 1, 'k': 'param1', 'config': {...}}

The documentation says:

locals() Update and return a dictionary representing the current local
symbol table. Free variables are returned by locals() when it is
called in function blocks, but not in class blocks. Note that at the
module level, locals() and globals() are the same dictionary.

I would have used "… return THE dictionary …" rather than "a", but, anyway…

Thank you all.

For people hitting this page in the future. I started using config=locals() for keeping a config-like dictionary after reading examples on authoritative sources, but that assignment is not "correct" and leads to these strange behaviours, very difficult to spot in complex code.

A better way to use locals() to keep a config dictionary is config = {**locals()}:

def f(param1, param2):
    config = {**locals()}
    print("1. config:")
    for k, v in config.items():
        print(f"- {k}: {v}")
        
    name = "hello"
    locals()
    
    print("2. config:")
    for k, v in config.items():
        print(f"- {k}: {v}")
    
f(1, 2)

with the expected, normal behaviour, especially for people used to languages with variables instead of names:

1. config:
- param2: 2
- param1: 1
2. config:
- param2: 2
- param1: 1

And I agree with the author of the accepted answer: documentation should be closer to the actual semantics.

Asked By: Antonio Sesto

||

Source

Answer 1

So, eval will retreive the locals() implicitly. From the docs for eval:

If the locals dictionary is omitted it defaults to the globals
dictionary. If both dictionaries are omitted, the expression is
executed with the globals and locals in the environment where eval()
is called.

And then from the docs on locals:

Update and return a dictionary representing the current local symbol table.

So the precise behavior is not well defined. But it seems it is re-using the same dictionary, and updating it from the actual local symbol table each time it is called.

So:

>>> def foo():
...     l = locals()
...     print(l)
...     a = 42
...     print(l is locals())
...     print(l)
...     b = 11
...     print(l)
...
>>> foo()
{}
True
{'l': {...}, 'a': 42}
{'l': {...}, 'a': 42}
>>>

Answered By: juanpa.arrivillaga

strange behaviour of locals() dictionaries when using eval

Question:

Answers: