How to inject variable into scope with a decorator?
Question:
[Disclaimer: there may be more pythonic ways of doing what I want to do, but I want to know how python’s scoping works here]
I’m trying to find a way to make a decorator that does something like injecting a name into the scope of another function (such that the name does not leak outside the decorator’s scope). For example, if I have a function that says to print a variable named var
that has not been defined, I would like to define it within a decorator where it is called. Here is an example that breaks:
c = 'Message'
def decorator_factory(value):
def msg_decorator(f):
def inner_dec(*args, **kwargs):
var = value
res = f(*args, **kwargs)
return res
return inner_dec
return msg_decorator
@decorator_factory(c)
def msg_printer():
print var
msg_printer()
I would like it to print “Message
“, but it gives:
NameError: global name 'var' is not defined
The traceback even points to wher var
is defined:
<ipython-input-25-34b84bee70dc> in inner_dec(*args, **kwargs)
8 def inner_dec(*args, **kwargs):
9 var = value
---> 10 res = f(*args, **kwargs)
11 return res
12 return inner_dec
So I don’t understand why it can’t find var
.
Is there any way to do something like this?
Answers:
You can’t. Scoped names (closures) are determined at compile time, you cannot add more at runtime.
The best you can hope to achieve is to add global names, using the function’s own global namespace:
def decorator_factory(value):
def msg_decorator(f):
def inner_dec(*args, **kwargs):
g = f.__globals__ # use f.func_globals for py < 2.6
sentinel = object()
oldvalue = g.get('var', sentinel)
g['var'] = value
try:
res = f(*args, **kwargs)
finally:
if oldvalue is sentinel:
del g['var']
else:
g['var'] = oldvalue
return res
return inner_dec
return msg_decorator
f.__globals__
is the global namespace for the wrapped function, so this works even if the decorator lives in a different module. If var
was defined as a global already, it is replaced with the new value, and after calling the function, the globals are restored.
This works because any name in a function that is not assigned to, and is not found in a surrounding scope, is marked as a global instead.
Demo:
>>> c = 'Message'
>>> @decorator_factory(c)
... def msg_printer():
... print var
...
>>> msg_printer()
Message
>>> 'var' in globals()
False
But instead of decorating, I could just as well have defined var
in the global scope directly.
Note that altering the globals is not thread safe, and any transient calls to other functions in the same module will also still see this same global.
You can’t. Python has lexical scoping. That means the meaning of an identifier is determined solely based on the scopes that physically surround it when you look at the source code.
Here is a simple demonstration of using a decorator to add a variable into the scope of a function.
>>> def add_name(name):
... def inner(func):
... # Same as defining name within wrapped
... # function.
... func.func_globals['name'] = name
...
... # Simply returns wrapped function reference.
... return func
...
... return inner
...
>>> @add_name("Bobby")
... def say_hello():
... print "Hello %s!" % name
...
>>> print say_hello()
Hello Bobby!
>>>
Python is lexically scoped, so I’m afraid there is no clean way to do what you want without some potentially nasty side effects. I recommend just passing var into the function via the decorator.
c = 'Message'
def decorator_factory(value):
def msg_decorator(f):
def inner_dec(*args, **kwargs):
res = f(value, *args, **kwargs)
return res
inner_dec.__name__ = f.__name__
inner_dec.__doc__ = f.__doc__
return inner_dec
return msg_decorator
@decorator_factory(c)
def msg_printer(var):
print var
msg_printer() # prints 'Message'
There is a clean way to do what you want without using global variable. If you want to be stateless and threads safe, you don’t really have the choice.
Use the “kwargs” variable:
c = 'Message'
def decorator_factory(value):
def msg_decorator(f):
def inner_dec(*args, **kwargs):
kwargs["var"] = value
res = f(*args, **kwargs)
return res
return inner_dec
return msg_decorator
@decorator_factory(c)
def msg_printer(*args, **kwargs):
print kwargs["var"]
msg_printer()
Here’s a way of injecting multiple variables into a function’s scope in a manner somewhat similar to what @Martijn Pieters does in his answer. I’m posting it primarily because it’s a more general solution and would not need to be applied multiple times to do it — as would be required by his (and many of the other) answers.
It should be noted that a closure is formed between the decorated function and the namespace
dictionary, so changing its contents — e.g. namespace['a'] = 42
— will affect subsequent calls to the function.
from functools import wraps
def inject_variables(context):
""" Decorator factory. """
def variable_injector(func):
""" Decorator. """
@wraps(func)
def decorator(*args, **kwargs):
func_globals = func.__globals__
# Save copy of any global values that will be replaced.
saved_values = {key: func_globals[key] for key in context
if key in func_globals}
func_globals.update(context)
try:
result = func(*args, **kwargs)
finally:
func_globals.update(saved_values) # Restore replaced globals.
return result
return decorator
return variable_injector
if __name__ == '__main__':
namespace = dict(a=5, b=3)
@inject_variables(namespace)
def test():
print('a:', a)
print('b:', b)
test()
Assuming that in python functions are objects, you can do…
#!/usr/bin/python3
class DecorClass(object):
def __init__(self, arg1, arg2):
self.a1 = arg1
self.a2 = arg2
def __call__(self, function):
def wrapped(*args):
print('inside class decorator >>')
print('class members: {0}, {1}'.format(self.a1, self.a2))
print('wrapped function: {}'.format(args))
function(*args, self.a1, self.a2)
return wrapped
@DecorClass(1, 2)
def my_function(f1, f2, *args):
print('inside decorated function >>')
print('decorated function arguments: {0}, {1}'.format(f1, f2))
print('decorator class args: {}'.format(args))
if __name__ == '__main__':
my_function(3, 4)
and the result is:
inside class decorator >>
class members: 1, 2
wrapped function: (3, 4)
inside decorated function >>
decorated function arguments: 3, 4
decorator class args: (1, 2)
more explanation here http://python-3-patterns-idioms-test.readthedocs.io/en/latest/PythonDecorators.html
def merge(d1, d2):
d = d1.copy()
d.update(d2)
return d
# A decorator to inject variables
def valueDecorator(*_args, **_kargs):
def wrapper(f):
def wrapper2(*args, **kargs):
return f(*args, **kargs)
wrapper2.__name__ = f.__name__
wrapper2.__doc__ = f.__doc__
oldVars = getattr(f, 'Vars', [])
oldNamedVars = getattr(f, 'NamedVars', {})
wrapper2.Vars = oldVars + list(_args)
wrapper2.NamedVars = merge(oldNamedVars, _kargs)
return wrapper2
return wrapper
@valueDecorator(12, 13, a=2)
@valueDecorator(10, 11, a=1)
def func():
print(func.Vars)
print(func.NamedVars)
Instead of revising the global scope, changing the annotated function itself is more reasonable.
Update __globals__
works for me.
def f():
print(a)
def with_context(**kw):
def deco(fn):
g = fn.__globals__
g.update(kw)
return fn
return deco
with_context(a=3)(f)() # 3
I have catched problem with solution using globals.
Context of globals may be overwritten when you have several concurrent requests. I thought that impossible, but it is – after some time I have catched change of context(globals) if request wasn’t quick.
Better solution is to pass variable using kwargs:
def is_login(old_fuction):
def new_function(request, *args, **kwargs):
secret_token = request.COOKIES.get('secret_token')
if secret_token:
items = SomeModel.objects.get(cookie = secret_token)
if len(items) > 0:
item = items[0]
kwargs['current_user'] = item
return old_fuction(request, *args, **kwargs)
else:
return HttpResponse('error')
return HttpResponse(status=404)
return new_function
@is_login
def some_func(request, current_user):
return HttpResponse(current_user.name)
You’ll have to add extra parameter to each decorated function.
I found an interesting post provides a different solution by creating functions on the fly. Basically:
def wrapper(func):
cust_globals = func.__globals__.copy()
# Update cust_globals to your liking
# Return a new function
return types.FunctionType(
func.__code__, cust_globals, func.__name__, func.__defaults__, func.__closure__
)
See https://hardenedapple.github.io/stories/computers/python_function_override/
[Disclaimer: there may be more pythonic ways of doing what I want to do, but I want to know how python’s scoping works here]
I’m trying to find a way to make a decorator that does something like injecting a name into the scope of another function (such that the name does not leak outside the decorator’s scope). For example, if I have a function that says to print a variable named var
that has not been defined, I would like to define it within a decorator where it is called. Here is an example that breaks:
c = 'Message'
def decorator_factory(value):
def msg_decorator(f):
def inner_dec(*args, **kwargs):
var = value
res = f(*args, **kwargs)
return res
return inner_dec
return msg_decorator
@decorator_factory(c)
def msg_printer():
print var
msg_printer()
I would like it to print “Message
“, but it gives:
NameError: global name 'var' is not defined
The traceback even points to wher var
is defined:
<ipython-input-25-34b84bee70dc> in inner_dec(*args, **kwargs)
8 def inner_dec(*args, **kwargs):
9 var = value
---> 10 res = f(*args, **kwargs)
11 return res
12 return inner_dec
So I don’t understand why it can’t find var
.
Is there any way to do something like this?
You can’t. Scoped names (closures) are determined at compile time, you cannot add more at runtime.
The best you can hope to achieve is to add global names, using the function’s own global namespace:
def decorator_factory(value):
def msg_decorator(f):
def inner_dec(*args, **kwargs):
g = f.__globals__ # use f.func_globals for py < 2.6
sentinel = object()
oldvalue = g.get('var', sentinel)
g['var'] = value
try:
res = f(*args, **kwargs)
finally:
if oldvalue is sentinel:
del g['var']
else:
g['var'] = oldvalue
return res
return inner_dec
return msg_decorator
f.__globals__
is the global namespace for the wrapped function, so this works even if the decorator lives in a different module. If var
was defined as a global already, it is replaced with the new value, and after calling the function, the globals are restored.
This works because any name in a function that is not assigned to, and is not found in a surrounding scope, is marked as a global instead.
Demo:
>>> c = 'Message'
>>> @decorator_factory(c)
... def msg_printer():
... print var
...
>>> msg_printer()
Message
>>> 'var' in globals()
False
But instead of decorating, I could just as well have defined var
in the global scope directly.
Note that altering the globals is not thread safe, and any transient calls to other functions in the same module will also still see this same global.
You can’t. Python has lexical scoping. That means the meaning of an identifier is determined solely based on the scopes that physically surround it when you look at the source code.
Here is a simple demonstration of using a decorator to add a variable into the scope of a function.
>>> def add_name(name):
... def inner(func):
... # Same as defining name within wrapped
... # function.
... func.func_globals['name'] = name
...
... # Simply returns wrapped function reference.
... return func
...
... return inner
...
>>> @add_name("Bobby")
... def say_hello():
... print "Hello %s!" % name
...
>>> print say_hello()
Hello Bobby!
>>>
Python is lexically scoped, so I’m afraid there is no clean way to do what you want without some potentially nasty side effects. I recommend just passing var into the function via the decorator.
c = 'Message'
def decorator_factory(value):
def msg_decorator(f):
def inner_dec(*args, **kwargs):
res = f(value, *args, **kwargs)
return res
inner_dec.__name__ = f.__name__
inner_dec.__doc__ = f.__doc__
return inner_dec
return msg_decorator
@decorator_factory(c)
def msg_printer(var):
print var
msg_printer() # prints 'Message'
There is a clean way to do what you want without using global variable. If you want to be stateless and threads safe, you don’t really have the choice.
Use the “kwargs” variable:
c = 'Message'
def decorator_factory(value):
def msg_decorator(f):
def inner_dec(*args, **kwargs):
kwargs["var"] = value
res = f(*args, **kwargs)
return res
return inner_dec
return msg_decorator
@decorator_factory(c)
def msg_printer(*args, **kwargs):
print kwargs["var"]
msg_printer()
Here’s a way of injecting multiple variables into a function’s scope in a manner somewhat similar to what @Martijn Pieters does in his answer. I’m posting it primarily because it’s a more general solution and would not need to be applied multiple times to do it — as would be required by his (and many of the other) answers.
It should be noted that a closure is formed between the decorated function and the namespace
dictionary, so changing its contents — e.g. namespace['a'] = 42
— will affect subsequent calls to the function.
from functools import wraps
def inject_variables(context):
""" Decorator factory. """
def variable_injector(func):
""" Decorator. """
@wraps(func)
def decorator(*args, **kwargs):
func_globals = func.__globals__
# Save copy of any global values that will be replaced.
saved_values = {key: func_globals[key] for key in context
if key in func_globals}
func_globals.update(context)
try:
result = func(*args, **kwargs)
finally:
func_globals.update(saved_values) # Restore replaced globals.
return result
return decorator
return variable_injector
if __name__ == '__main__':
namespace = dict(a=5, b=3)
@inject_variables(namespace)
def test():
print('a:', a)
print('b:', b)
test()
Assuming that in python functions are objects, you can do…
#!/usr/bin/python3
class DecorClass(object):
def __init__(self, arg1, arg2):
self.a1 = arg1
self.a2 = arg2
def __call__(self, function):
def wrapped(*args):
print('inside class decorator >>')
print('class members: {0}, {1}'.format(self.a1, self.a2))
print('wrapped function: {}'.format(args))
function(*args, self.a1, self.a2)
return wrapped
@DecorClass(1, 2)
def my_function(f1, f2, *args):
print('inside decorated function >>')
print('decorated function arguments: {0}, {1}'.format(f1, f2))
print('decorator class args: {}'.format(args))
if __name__ == '__main__':
my_function(3, 4)
and the result is:
inside class decorator >>
class members: 1, 2
wrapped function: (3, 4)
inside decorated function >>
decorated function arguments: 3, 4
decorator class args: (1, 2)
more explanation here http://python-3-patterns-idioms-test.readthedocs.io/en/latest/PythonDecorators.html
def merge(d1, d2):
d = d1.copy()
d.update(d2)
return d
# A decorator to inject variables
def valueDecorator(*_args, **_kargs):
def wrapper(f):
def wrapper2(*args, **kargs):
return f(*args, **kargs)
wrapper2.__name__ = f.__name__
wrapper2.__doc__ = f.__doc__
oldVars = getattr(f, 'Vars', [])
oldNamedVars = getattr(f, 'NamedVars', {})
wrapper2.Vars = oldVars + list(_args)
wrapper2.NamedVars = merge(oldNamedVars, _kargs)
return wrapper2
return wrapper
@valueDecorator(12, 13, a=2)
@valueDecorator(10, 11, a=1)
def func():
print(func.Vars)
print(func.NamedVars)
Instead of revising the global scope, changing the annotated function itself is more reasonable.
Update __globals__
works for me.
def f():
print(a)
def with_context(**kw):
def deco(fn):
g = fn.__globals__
g.update(kw)
return fn
return deco
with_context(a=3)(f)() # 3
I have catched problem with solution using globals.
Context of globals may be overwritten when you have several concurrent requests. I thought that impossible, but it is – after some time I have catched change of context(globals) if request wasn’t quick.
Better solution is to pass variable using kwargs:
def is_login(old_fuction):
def new_function(request, *args, **kwargs):
secret_token = request.COOKIES.get('secret_token')
if secret_token:
items = SomeModel.objects.get(cookie = secret_token)
if len(items) > 0:
item = items[0]
kwargs['current_user'] = item
return old_fuction(request, *args, **kwargs)
else:
return HttpResponse('error')
return HttpResponse(status=404)
return new_function
@is_login
def some_func(request, current_user):
return HttpResponse(current_user.name)
You’ll have to add extra parameter to each decorated function.
I found an interesting post provides a different solution by creating functions on the fly. Basically:
def wrapper(func):
cust_globals = func.__globals__.copy()
# Update cust_globals to your liking
# Return a new function
return types.FunctionType(
func.__code__, cust_globals, func.__name__, func.__defaults__, func.__closure__
)
See https://hardenedapple.github.io/stories/computers/python_function_override/