Can a decorated function access variables of the decorator
Question:
I’m trying to understand how decorators work, and was wondering if a decorated function can access variables of the decorator. For example, in the following code, how do I make f1 have access to localVariable? Is that possible, and is it even a good way of doing things?
def funcDec(func):
localVariable = "I'm a local string"
def func2Return(*args):
print "Calling localVariable from decorator " + localVariable
func(*args)
print "done with calling f1"
return func2Return
@funcDec
def f1(x, y):
print x + y
print localVariable
f1(2, 3)
Answers:
No, you can’t. See this previous question. Just because the function is a decorator doesn’t mean functions it calls have special access to its variables. If you do this:
def func():
a = 2
otherFunc()
Then otherFunc doesn’t have access to the variable a. That’s how it works for all function calls, and it’s how it works for decorators too.
Now, the wrapper function you define inside the decorator (func2Return in your example) does have access to the variables, because that function is lexically in the same scope as those variables. So your line print "Calling localVariable from decorator " + localVariable will work. You can use this to some extent to wrap the decorated function with behavior that depends on variables in the decorator. But the function actually being decorated (f1 in your example) doesn’t have access to those variables.
A function only has access to local variables from the scope where the function definition actually is. Functions don’t get variables from calling scopes. (This is a good thing. If they did, it would be a huge mess.)
I think it helps if you keep in mind that a decorator
@deco
def f(...): ...
is just syntactic sugar for
def f(...): ...
f = deco(f)
rather than some kind of macro expansion. In Python the scope of a variable is determined statically, so for a global (module-level) function a variable that is neither passed as an argument nor assigned to will be looked up in the global namespace.
Therefore you have to pass on a local variable of func2Return() explicitly. Change the signature of f1 to f1(x, y, localvariable=None) and have the wrapper function fun2Return call it with
f1(*args, localvariable=localvariable)
Because of Python’s scoping rules, a decorated function generally can’t access any variables in the decorator. However, since functions can have arbitrary attributes assigned to them, you could do something like the following in the decorator to get a similar effect (due to the same scoping rules):
def funcDec(func):
localVariable = "I'm a local string"
def wrapped(*args):
print("Calling localVariable from funcDec " + localVariable)
func(*args)
print("done with calling f1")
wrapped.attrib = localVariable
return wrapped
@funcDec
def f1(x, y):
print(x + y)
print('f1.attrib: {!r}'.format(f1.attrib))
f1(2, 3)
Which would produce the following output:
Calling localVariable from funcDec I'm a local string
5
f1.attrib: "I'm a local string"
done with calling f1
Someone asked whether this could be applied to methods of a class:
The answer is "yes", but you have to reference the method either through the class itself or the instance of it passed as the self argument. Both techniques are shown below. Using self is preferable since it makes the code independent of the name of the class it’s is in.
class Test(object):
@funcDec
def f1(self):
print('{}.f1() called'.format(self.__class__.__name__))
print('self.f1.attrib: {!r}'.format(self.f1.attrib)) # Preferred.
print('Test.f1.attrib: {!r}'.format(Test.f1.attrib)) # Also works.
print()
test = Test()
test.f1()
Output:
Calling localVariable from funcDec I'm a local string
Test.f1() called
self.f1.attrib: "I'm a local string"
Test.f1.attrib: "I'm a local string"
done with calling f1
Update
Another way of doing this that would give the decorated function more direct access to decorator variables would be to temporarily "inject" them into its global namespace (and then remove them afterwards).
I got the idea from @Martijn Pieters’ answer to the somewhat related question: How to inject variable into scope with a decorator?
def funcDec(func):
localVariable = "I'm a local string"
# Local variable(s) to inject into wrapped func.
context = {'localVariable': localVariable}
def wrapped(*args):
func_globals = func.__globals__
# Save copy of any global values that will be replaced.
saved_values = {key: func_globals[key] for key in context if key in func_globals}
func_globals.update(context)
print(f'Calling localVariable from funcDec: {localVariable!r}')
try:
func(*args)
finally:
func_globals.update(saved_values) # Restore any replaced globals.
print(f'done with calling {func.__name__}()')
return wrapped
@funcDec
def f1(x, y):
print(x + y)
print(f'Calling funcDec localVariable from f1: {localVariable!r}')
f1(2, 3)
Result from this version:
Calling localVariable from funcDec: "I'm a local string"
5
Calling funcDec localVariable from f1: "I'm a local string"
done with calling f1()
I found this question trying to use variables in (I think) the opposite direction, and found the answer from Martineau to apply in that opposite direction. That is, I am using a decorator to wrap some standard error handling around a series of similar functions but want to pass some of that error info up since that’s where my logging is.
So, I was able to do something like this:
def error_handler(func):
def wrapper_decorator(*args,**kwargs):
try:
func(*args,**kwargs)
except Exception as e: # I actually handle things better than a general grab here
logger.error(f'func had bad value {error_handler.value}')
return wrapper_decorator
@error_handler
def main():
# blah blah, do stuff, read a dataframe from a web page
error_handler.value = value # value is created within this main() func
# blah blah, do more stuff where we actually end up having an error
This sets the .value attribute on error_handler to a value that is generated within the scope of main(), effectively letting me ‘pass up’ that value.
In my actual specific example, I’m using a dataframe and getting errors when I save to serve because the data is poorly formatted. main() reads in the dataframe, transforms it, and (tries) to save to server. This setup allows me to pass the dataframe (as the value in the example code) to my actual error_handler function and then save the dataframe as a csv so I can inspect it, and I don’t have to have the saving of the dataframe within each individual function.
I’m trying to understand how decorators work, and was wondering if a decorated function can access variables of the decorator. For example, in the following code, how do I make f1 have access to localVariable? Is that possible, and is it even a good way of doing things?
def funcDec(func):
localVariable = "I'm a local string"
def func2Return(*args):
print "Calling localVariable from decorator " + localVariable
func(*args)
print "done with calling f1"
return func2Return
@funcDec
def f1(x, y):
print x + y
print localVariable
f1(2, 3)
No, you can’t. See this previous question. Just because the function is a decorator doesn’t mean functions it calls have special access to its variables. If you do this:
def func():
a = 2
otherFunc()
Then otherFunc doesn’t have access to the variable a. That’s how it works for all function calls, and it’s how it works for decorators too.
Now, the wrapper function you define inside the decorator (func2Return in your example) does have access to the variables, because that function is lexically in the same scope as those variables. So your line print "Calling localVariable from decorator " + localVariable will work. You can use this to some extent to wrap the decorated function with behavior that depends on variables in the decorator. But the function actually being decorated (f1 in your example) doesn’t have access to those variables.
A function only has access to local variables from the scope where the function definition actually is. Functions don’t get variables from calling scopes. (This is a good thing. If they did, it would be a huge mess.)
I think it helps if you keep in mind that a decorator
@deco
def f(...): ...
is just syntactic sugar for
def f(...): ...
f = deco(f)
rather than some kind of macro expansion. In Python the scope of a variable is determined statically, so for a global (module-level) function a variable that is neither passed as an argument nor assigned to will be looked up in the global namespace.
Therefore you have to pass on a local variable of func2Return() explicitly. Change the signature of f1 to f1(x, y, localvariable=None) and have the wrapper function fun2Return call it with
f1(*args, localvariable=localvariable)
Because of Python’s scoping rules, a decorated function generally can’t access any variables in the decorator. However, since functions can have arbitrary attributes assigned to them, you could do something like the following in the decorator to get a similar effect (due to the same scoping rules):
def funcDec(func):
localVariable = "I'm a local string"
def wrapped(*args):
print("Calling localVariable from funcDec " + localVariable)
func(*args)
print("done with calling f1")
wrapped.attrib = localVariable
return wrapped
@funcDec
def f1(x, y):
print(x + y)
print('f1.attrib: {!r}'.format(f1.attrib))
f1(2, 3)
Which would produce the following output:
Calling localVariable from funcDec I'm a local string
5
f1.attrib: "I'm a local string"
done with calling f1
Someone asked whether this could be applied to methods of a class:
The answer is "yes", but you have to reference the method either through the class itself or the instance of it passed as the self argument. Both techniques are shown below. Using self is preferable since it makes the code independent of the name of the class it’s is in.
class Test(object):
@funcDec
def f1(self):
print('{}.f1() called'.format(self.__class__.__name__))
print('self.f1.attrib: {!r}'.format(self.f1.attrib)) # Preferred.
print('Test.f1.attrib: {!r}'.format(Test.f1.attrib)) # Also works.
print()
test = Test()
test.f1()
Output:
Calling localVariable from funcDec I'm a local string
Test.f1() called
self.f1.attrib: "I'm a local string"
Test.f1.attrib: "I'm a local string"
done with calling f1
Update
Another way of doing this that would give the decorated function more direct access to decorator variables would be to temporarily "inject" them into its global namespace (and then remove them afterwards).
I got the idea from @Martijn Pieters’ answer to the somewhat related question: How to inject variable into scope with a decorator?
def funcDec(func):
localVariable = "I'm a local string"
# Local variable(s) to inject into wrapped func.
context = {'localVariable': localVariable}
def wrapped(*args):
func_globals = func.__globals__
# Save copy of any global values that will be replaced.
saved_values = {key: func_globals[key] for key in context if key in func_globals}
func_globals.update(context)
print(f'Calling localVariable from funcDec: {localVariable!r}')
try:
func(*args)
finally:
func_globals.update(saved_values) # Restore any replaced globals.
print(f'done with calling {func.__name__}()')
return wrapped
@funcDec
def f1(x, y):
print(x + y)
print(f'Calling funcDec localVariable from f1: {localVariable!r}')
f1(2, 3)
Result from this version:
Calling localVariable from funcDec: "I'm a local string"
5
Calling funcDec localVariable from f1: "I'm a local string"
done with calling f1()
I found this question trying to use variables in (I think) the opposite direction, and found the answer from Martineau to apply in that opposite direction. That is, I am using a decorator to wrap some standard error handling around a series of similar functions but want to pass some of that error info up since that’s where my logging is.
So, I was able to do something like this:
def error_handler(func):
def wrapper_decorator(*args,**kwargs):
try:
func(*args,**kwargs)
except Exception as e: # I actually handle things better than a general grab here
logger.error(f'func had bad value {error_handler.value}')
return wrapper_decorator
@error_handler
def main():
# blah blah, do stuff, read a dataframe from a web page
error_handler.value = value # value is created within this main() func
# blah blah, do more stuff where we actually end up having an error
This sets the .value attribute on error_handler to a value that is generated within the scope of main(), effectively letting me ‘pass up’ that value.
In my actual specific example, I’m using a dataframe and getting errors when I save to serve because the data is poorly formatted. main() reads in the dataframe, transforms it, and (tries) to save to server. This setup allows me to pass the dataframe (as the value in the example code) to my actual error_handler function and then save the dataframe as a csv so I can inspect it, and I don’t have to have the saving of the dataframe within each individual function.