In Python 2, how do I write to variable in the parent scope?
Question:
I have some code like:
def example():
# other logic omitted
stored_blocks = {}
def replace_blocks(m):
block = m.group(0)
block_hash = sha1(block)
stored_blocks[block_hash] = block
return '{{{%s}}}' % block_hash
num_converted = 0
def convert_variables(m):
name = m.group(1)
num_converted += 1
return '<%%= %s %%>' % name
fixed = MATCH_DECLARE_NEW.sub('', template)
fixed = MATCH_PYTHON_BLOCK.sub(replace_blocks, fixed)
fixed = MATCH_FORMAT.sub(convert_variables, fixed)
# more logic...
Adding elements to stored_blocks
works fine, but I cannot increase num_converted
in the second nested function. I get an exception that says UnboundLocalError: local variable 'num_converted' referenced before assignment
.
I know that in 3.x, I could try nonlocal num_converted
, but how can I solve the problem in 2.x? I don’t want to use a global variable for this.
Answers:
Using the global
keyword is fine. If you write:
num_converted = 0
def convert_variables(m):
global num_converted
name = m.group(1)
num_converted += 1
return '<%%= %s %%>' % name
… num_converted
doesn’t become a “global variable” (i.e. it doesn’t become visible in any other unexpected places), it just means it can be modified inside convert_variables
. That seems to be exactly what you want.
To put it another way, num_converted
is already a global variable. All the global num_converted
syntax does is tell Python “inside this function, don’t create a local num_converted
variable, instead, use the existing global one.
Problem: This is because Python’s scoping rules are demented. The presence of the +=
assignment operator marks the target, num_converted
, as local to the enclosing function’s scope, and there is no sound way in Python 2.x to access just one scoping level out from there. Only the global
keyword can lift variable references out of the current scope, and it takes you straight to the top.
Fix: Turn num_converted
into a single-element array.
num_converted = [0]
def convert_variables(m):
name = m.group(1)
num_converted[0] += 1
return '<%%= %s %%>' % name
What about using a class instance to hold the state?
You instantiate a class and pass instance methods to subs and those functions would have a reference to self…
(see below for the edited answer)
You can use something like:
def convert_variables(m):
name = m.group(1)
convert_variables.num_converted += 1
return '<%%= %s %%>' % name
convert_variables.num_converted = 0
This way, num_converted
works as a C-like “static” variable of the convert_variable method
(edited)
def convert_variables(m):
name = m.group(1)
convert_variables.num_converted = convert_variables.__dict__.get("num_converted", 0) + 1
return '<%%= %s %%>' % name
This way, you don’t need to initialize the counter in the main procedure.
I have couple of remarks.
First, one application for such nested functions comes up when dealing with raw callbacks, as are used in libraries like xml.parsers.expat. (That the library authors chose this approach may be objectionable, but … there are reasons to use it nonetheless.)
Second: within a class, there are much nicer alternatives to the array (num_converted[0]). I suppose this is what Sebastjan was talking about.
class MainClass:
_num_converted = 0
def outer_method( self ):
def convert_variables(m):
name = m.group(1)
self._num_converted += 1
return '<%%= %s %%>' % name
It’s still odd enough to merit a comment in the code…
But the variable is at least local to the class.
Modified from: https://stackoverflow.com/a/40690954/819544
You can leverage the inspect
module to access the calling scope’s globals dict and write into that. That means this trick can even be leveraged to access the calling scope from a nested function defined in an imported submodule.
import inspect
def get_globals(scope_level=0):
return dict(inspect.getmembers(inspect.stack()[scope_level][0]))["f_globals"]
num_converted = 0
def foobar():
get_globals(0)['num_converted'] += 1
foobar()
print(num_converted)
# 1
Play with the scope_level
argument as needed. Setting scope_level=1
works for a function defined in a submodule, scope_level=2
for the inner function defined in a decorator in a submodule, etc.
NB: Just because you can do this, doesn’t mean you should.
I have some code like:
def example():
# other logic omitted
stored_blocks = {}
def replace_blocks(m):
block = m.group(0)
block_hash = sha1(block)
stored_blocks[block_hash] = block
return '{{{%s}}}' % block_hash
num_converted = 0
def convert_variables(m):
name = m.group(1)
num_converted += 1
return '<%%= %s %%>' % name
fixed = MATCH_DECLARE_NEW.sub('', template)
fixed = MATCH_PYTHON_BLOCK.sub(replace_blocks, fixed)
fixed = MATCH_FORMAT.sub(convert_variables, fixed)
# more logic...
Adding elements to stored_blocks
works fine, but I cannot increase num_converted
in the second nested function. I get an exception that says UnboundLocalError: local variable 'num_converted' referenced before assignment
.
I know that in 3.x, I could try nonlocal num_converted
, but how can I solve the problem in 2.x? I don’t want to use a global variable for this.
Using the global
keyword is fine. If you write:
num_converted = 0
def convert_variables(m):
global num_converted
name = m.group(1)
num_converted += 1
return '<%%= %s %%>' % name
… num_converted
doesn’t become a “global variable” (i.e. it doesn’t become visible in any other unexpected places), it just means it can be modified inside convert_variables
. That seems to be exactly what you want.
To put it another way, num_converted
is already a global variable. All the global num_converted
syntax does is tell Python “inside this function, don’t create a local num_converted
variable, instead, use the existing global one.
Problem: This is because Python’s scoping rules are demented. The presence of the +=
assignment operator marks the target, num_converted
, as local to the enclosing function’s scope, and there is no sound way in Python 2.x to access just one scoping level out from there. Only the global
keyword can lift variable references out of the current scope, and it takes you straight to the top.
Fix: Turn num_converted
into a single-element array.
num_converted = [0]
def convert_variables(m):
name = m.group(1)
num_converted[0] += 1
return '<%%= %s %%>' % name
What about using a class instance to hold the state?
You instantiate a class and pass instance methods to subs and those functions would have a reference to self…
(see below for the edited answer)
You can use something like:
def convert_variables(m):
name = m.group(1)
convert_variables.num_converted += 1
return '<%%= %s %%>' % name
convert_variables.num_converted = 0
This way, num_converted
works as a C-like “static” variable of the convert_variable method
(edited)
def convert_variables(m):
name = m.group(1)
convert_variables.num_converted = convert_variables.__dict__.get("num_converted", 0) + 1
return '<%%= %s %%>' % name
This way, you don’t need to initialize the counter in the main procedure.
I have couple of remarks.
First, one application for such nested functions comes up when dealing with raw callbacks, as are used in libraries like xml.parsers.expat. (That the library authors chose this approach may be objectionable, but … there are reasons to use it nonetheless.)
Second: within a class, there are much nicer alternatives to the array (num_converted[0]). I suppose this is what Sebastjan was talking about.
class MainClass:
_num_converted = 0
def outer_method( self ):
def convert_variables(m):
name = m.group(1)
self._num_converted += 1
return '<%%= %s %%>' % name
It’s still odd enough to merit a comment in the code…
But the variable is at least local to the class.
Modified from: https://stackoverflow.com/a/40690954/819544
You can leverage the inspect
module to access the calling scope’s globals dict and write into that. That means this trick can even be leveraged to access the calling scope from a nested function defined in an imported submodule.
import inspect
def get_globals(scope_level=0):
return dict(inspect.getmembers(inspect.stack()[scope_level][0]))["f_globals"]
num_converted = 0
def foobar():
get_globals(0)['num_converted'] += 1
foobar()
print(num_converted)
# 1
Play with the scope_level
argument as needed. Setting scope_level=1
works for a function defined in a submodule, scope_level=2
for the inner function defined in a decorator in a submodule, etc.
NB: Just because you can do this, doesn’t mean you should.