Modify bound variables of a closure in Python
Question:
Is there any way to modify the bound value of one of the variables inside a closure? Look at the example to understand it better.
def foo():
var_a = 2
var_b = 3
def _closure(x):
return var_a + var_b + x
return _closure
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
# ...but what magic? Is this even possible?
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
Answers:
Why not make var_a and var_b arguments of the function foo?
def foo(var_a = 2, var_b = 3):
def _closure(x):
return var_a + var_b + x
return _closure
localClosure = foo() # uses default arguments 2, 3
print localClosure(1) # 2 + 3 + 1 = 6
localClosure = foo(0, 3)
print localClosure(1) # 0 + 3 + 1 = 4
I don’t think there is any way to do that in Python. When the closure is defined, the current state of variables in the enclosing scope is captured and no longer has a directly referenceable name (from outside the closure). If you were to call foo()
again, the new closure would have a different set of variables from the enclosing scope.
In your simple example, you might be better off using a class:
class foo:
def __init__(self):
self.var_a = 2
self.var_b = 3
def __call__(self, x):
return self.var_a + self.var_b + x
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
# ...but what magic? Is this even possible?
localClosure.var_a = 0
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
If you do use this technique I would no longer use the name localClosure
because it is no longer actually a closure. However, it works the same as one.
I’ve found an alternate answer answer to Greg’s, slightly less verbose because it uses Python 2.1’s custom function attributes (which conveniently enough can be accessed from inside their own function).
def foo():
var_b = 3
def _closure(x):
return _closure.var_a + var_b + x
_closure.func_dict['var_a'] = 2
return _closure
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
# ...but what magic? Is this even possible?
# apparently, it is
localClosure.var_a = 0
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
Thought I’d post it for completeness. Cheers anyways.
It is quite possible in python 3 thanks to the magic of nonlocal.
def foo():
var_a = 2
var_b = 3
def _closure(x, magic = None):
nonlocal var_a
if magic is not None:
var_a = magic
return var_a + var_b + x
return _closure
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
print(a)
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
localClosure(0, 0)
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
print(b)
We’ve done the following. I think it’s simpler than other solutions here.
class State:
pass
def foo():
st = State()
st.var_a = 2
st.var_b = 3
def _closure(x):
return st.var_a + st.var_b + x
def _set_a(a):
st.var_a = a
return _closure, _set_a
localClosure, localSetA = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
localSetA(0)
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
print a, b
I worked around a similar limitation by using one-item lists instead of a plain variable. It’s ugly but it works because modifying a list item doesn’t get treated as a binding operation by the interpreter.
For example:
def my_function()
max_value = [0]
def callback (data)
if (data.val > max_value[0]):
max_value[0] = data.val
# more code here
# . . .
results = some_function (callback)
store_max (max_value[0])
def foo():
var_a = 2
var_b = 3
def _closure(x):
return var_a + var_b + x
return _closure
def bar():
var_a = [2]
var_b = [3]
def _closure(x):
return var_a[0] + var_b[0] + x
def _magic(y):
var_a[0] = y
return _closure, _magic
localClosureFoo = foo()
a = localClosureFoo(1)
print a
localClosureBar, localClosureBarMAGIC = bar()
b = localClosureBar(1)
print b
localClosureBarMAGIC(0)
b = localClosureBar(1)
print b
slightly different from what was asked, but you could do:
def f():
a = 1
b = 2
def g(x, a=a, b=b):
return a + b + x
return g
h = f()
print(h(0))
print(h(0,2,3))
print(h(0))
and make the closure the default, to be overridden when needed.
Maybe there’s a further approach (even if it seems to be some years too late for my proposal 🙂
def foo():
def _closure(x):
return _closure.var_a + _closure.var_b + x
_closure.var_a = 2
_closure.var_b = 3
return _closure
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
print(a)
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
# ...but what magic? Is this even possible?
localClosure.var_a = 0
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
print(b)
From my point of view the class solution proposed is easier to read. But if you try to modiy a free variable inside a decorator this solution might come in handy: In comparison to a class based solution it’s easier to work with functools.wraps to preserve the meta data of the decorated function.
Question
Is there any way to modify the bound value of one of the variables inside a closure?
TLDR
Yes, this is possible starting in Python 3.7.0 alpha 1:
localClosure.__closure__[0].cell_contents = 0
Details
In Python, a closure remembers the variables from the scope in which it was defined by using a special __closure__
attribute. The __closure__
attribute is a tuple of cell
objects representing the variables from the outer scope, and the values of those variables are stored in the cell_contents
attribute of each cell
.
Given the code from the question, this can be seen by running the following:
# print the list of cells
print(localClosure.__closure__)
# (<cell at 0x7f941ca27a00: int object at 0x7f941a621950>, <cell at 0x7f941ca27eb0: int object at 0x7f941a621970>)
# print the values in the cells
print(', '.join(str(cell.cell_contents) for cell in localClosure.__closure__))
# 2, 3
# print the value in the first cell (var_a)
print(localClosure.__closure__[0].cell_contents)
# 2
The cell_contents
attribute of the cell
objects first became writable with bpo-30486 which was first included in Python 3.7.0 alpha 1
Complete working example:
def foo():
var_a = 2
var_b = 3
def _closure(x):
return var_a + var_b + x
return _closure
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
# ...but what magic? Is this even possible?
# the magic
# this changes the value in the cell representing var_a to be 0
localClosure.__closure__[0].cell_contents = 0
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 + 1 == 4
Is there any way to modify the bound value of one of the variables inside a closure? Look at the example to understand it better.
def foo():
var_a = 2
var_b = 3
def _closure(x):
return var_a + var_b + x
return _closure
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
# ...but what magic? Is this even possible?
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
Why not make var_a and var_b arguments of the function foo?
def foo(var_a = 2, var_b = 3):
def _closure(x):
return var_a + var_b + x
return _closure
localClosure = foo() # uses default arguments 2, 3
print localClosure(1) # 2 + 3 + 1 = 6
localClosure = foo(0, 3)
print localClosure(1) # 0 + 3 + 1 = 4
I don’t think there is any way to do that in Python. When the closure is defined, the current state of variables in the enclosing scope is captured and no longer has a directly referenceable name (from outside the closure). If you were to call foo()
again, the new closure would have a different set of variables from the enclosing scope.
In your simple example, you might be better off using a class:
class foo:
def __init__(self):
self.var_a = 2
self.var_b = 3
def __call__(self, x):
return self.var_a + self.var_b + x
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
# ...but what magic? Is this even possible?
localClosure.var_a = 0
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
If you do use this technique I would no longer use the name localClosure
because it is no longer actually a closure. However, it works the same as one.
I’ve found an alternate answer answer to Greg’s, slightly less verbose because it uses Python 2.1’s custom function attributes (which conveniently enough can be accessed from inside their own function).
def foo():
var_b = 3
def _closure(x):
return _closure.var_a + var_b + x
_closure.func_dict['var_a'] = 2
return _closure
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
# ...but what magic? Is this even possible?
# apparently, it is
localClosure.var_a = 0
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
Thought I’d post it for completeness. Cheers anyways.
It is quite possible in python 3 thanks to the magic of nonlocal.
def foo():
var_a = 2
var_b = 3
def _closure(x, magic = None):
nonlocal var_a
if magic is not None:
var_a = magic
return var_a + var_b + x
return _closure
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
print(a)
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
localClosure(0, 0)
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
print(b)
We’ve done the following. I think it’s simpler than other solutions here.
class State:
pass
def foo():
st = State()
st.var_a = 2
st.var_b = 3
def _closure(x):
return st.var_a + st.var_b + x
def _set_a(a):
st.var_a = a
return _closure, _set_a
localClosure, localSetA = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
localSetA(0)
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
print a, b
I worked around a similar limitation by using one-item lists instead of a plain variable. It’s ugly but it works because modifying a list item doesn’t get treated as a binding operation by the interpreter.
For example:
def my_function()
max_value = [0]
def callback (data)
if (data.val > max_value[0]):
max_value[0] = data.val
# more code here
# . . .
results = some_function (callback)
store_max (max_value[0])
def foo():
var_a = 2
var_b = 3
def _closure(x):
return var_a + var_b + x
return _closure
def bar():
var_a = [2]
var_b = [3]
def _closure(x):
return var_a[0] + var_b[0] + x
def _magic(y):
var_a[0] = y
return _closure, _magic
localClosureFoo = foo()
a = localClosureFoo(1)
print a
localClosureBar, localClosureBarMAGIC = bar()
b = localClosureBar(1)
print b
localClosureBarMAGIC(0)
b = localClosureBar(1)
print b
slightly different from what was asked, but you could do:
def f():
a = 1
b = 2
def g(x, a=a, b=b):
return a + b + x
return g
h = f()
print(h(0))
print(h(0,2,3))
print(h(0))
and make the closure the default, to be overridden when needed.
Maybe there’s a further approach (even if it seems to be some years too late for my proposal 🙂
def foo():
def _closure(x):
return _closure.var_a + _closure.var_b + x
_closure.var_a = 2
_closure.var_b = 3
return _closure
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
print(a)
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
# ...but what magic? Is this even possible?
localClosure.var_a = 0
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 +1 == 4
print(b)
From my point of view the class solution proposed is easier to read. But if you try to modiy a free variable inside a decorator this solution might come in handy: In comparison to a class based solution it’s easier to work with functools.wraps to preserve the meta data of the decorated function.
Question
Is there any way to modify the bound value of one of the variables inside a closure?
TLDR
Yes, this is possible starting in Python 3.7.0 alpha 1:
localClosure.__closure__[0].cell_contents = 0
Details
In Python, a closure remembers the variables from the scope in which it was defined by using a special __closure__
attribute. The __closure__
attribute is a tuple of cell
objects representing the variables from the outer scope, and the values of those variables are stored in the cell_contents
attribute of each cell
.
Given the code from the question, this can be seen by running the following:
# print the list of cells
print(localClosure.__closure__)
# (<cell at 0x7f941ca27a00: int object at 0x7f941a621950>, <cell at 0x7f941ca27eb0: int object at 0x7f941a621970>)
# print the values in the cells
print(', '.join(str(cell.cell_contents) for cell in localClosure.__closure__))
# 2, 3
# print the value in the first cell (var_a)
print(localClosure.__closure__[0].cell_contents)
# 2
The cell_contents
attribute of the cell
objects first became writable with bpo-30486 which was first included in Python 3.7.0 alpha 1
Complete working example:
def foo():
var_a = 2
var_b = 3
def _closure(x):
return var_a + var_b + x
return _closure
localClosure = foo()
# Local closure is now "return 2 + 3 + x"
a = localClosure(1) # 2 + 3 + 1 == 6
# DO SOME MAGIC HERE TO TURN "var_a" of the closure into 0
# ...but what magic? Is this even possible?
# the magic
# this changes the value in the cell representing var_a to be 0
localClosure.__closure__[0].cell_contents = 0
# Local closure is now "return 0 + 3 + x"
b = localClosure(1) # 0 + 3 + 1 == 4