Python class decorator arguments
Question:
I’m trying to pass optional arguments to my class decorator in python.
Below the code I currently have:
class Cache(object):
def __init__(self, function, max_hits=10, timeout=5):
self.function = function
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, *args):
# Here the code returning the correct thing.
@Cache
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2
The second decorator with arguments to overwrite the default one (max_hits=10, timeout=5
in my __init__
function), is not working and I got the exception TypeError: __init__() takes at least 2 arguments (3 given)
. I tried many solutions and read articles about it, but here I still can’t make it work.
Any idea to resolve this? Thanks!
Answers:
@Cache(max_hits=100, timeout=50)
calls __init__(max_hits=100, timeout=50)
, so you aren’t satisfying the function
argument.
You could implement your decorator via a wrapper method that detected whether a function was present. If it finds a function, it can return the Cache object. Otherwise, it can return a wrapper function that will be used as the decorator.
class _Cache(object):
def __init__(self, function, max_hits=10, timeout=5):
self.function = function
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, *args):
# Here the code returning the correct thing.
# wrap _Cache to allow for deferred calling
def Cache(function=None, max_hits=10, timeout=5):
if function:
return _Cache(function)
else:
def wrapper(function):
return _Cache(function, max_hits, timeout)
return wrapper
@Cache
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2
@Cache
def double(...):
...
is equivalent to
def double(...):
...
double=Cache(double)
While
@Cache(max_hits=100, timeout=50)
def double(...):
...
is equivalent to
def double(...):
...
double = Cache(max_hits=100, timeout=50)(double)
Cache(max_hits=100, timeout=50)(double)
has very different semantics than Cache(double)
.
It’s unwise to try to make Cache
handle both use cases.
You could instead use a decorator factory that can take optional max_hits
and timeout
arguments, and returns a decorator:
class Cache(object):
def __init__(self, function, max_hits=10, timeout=5):
self.function = function
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, *args):
# Here the code returning the correct thing.
def cache_hits(max_hits=10, timeout=5):
def _cache(function):
return Cache(function,max_hits,timeout)
return _cache
@cache_hits()
def double(x):
return x * 2
@cache_hits(max_hits=100, timeout=50)
def double(x):
return x * 2
PS. If the class Cache
has no other methods besides __init__
and __call__
, you can probably move all the code inside the _cache
function and eliminate Cache
altogether.
I’ve learned a lot from this question, thanks all. Isn’t the answer just to put empty brackets on the first @Cache
? Then you can move the function
parameter to __call__
.
class Cache(object):
def __init__(self, max_hits=10, timeout=5):
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, function, *args):
# Here the code returning the correct thing.
@Cache()
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2
Although I think this approach is simpler and more concise:
def cache(max_hits=10, timeout=5):
def caching_decorator(fn):
def decorated_fn(*args ,**kwargs):
# Here the code returning the correct thing.
return decorated_fn
return decorator
If you forget the parentheses when using the decorator, unfortunately you still don’t get an error until runtime, as the outer decorator parameters are passed the function you’re trying to decorate. Then at runtime the inner decorator complains:
TypeError: caching_decorator() takes exactly 1 argument (0 given).
However you can catch this, if you know your decorator’s parameters are never going to be a callable:
def cache(max_hits=10, timeout=5):
assert not callable(max_hits), "@cache passed a callable - did you forget to parenthesize?"
def caching_decorator(fn):
def decorated_fn(*args ,**kwargs):
# Here the code returning the correct thing.
return decorated_fn
return decorator
If you now try:
@cache
def some_method()
pass
You get an AssertionError
on declaration.
On a total tangent, I came across this post looking for decorators that decorate classes, rather than classes that decorate. In case anyone else does too, this question is useful.
I’d rather to include the wrapper inside the class’s __call__
method:
UPDATE:
This method has been tested in python 3.6, so I’m not sure about the higher or earlier versions.
class Cache:
def __init__(self, max_hits=10, timeout=5):
# Remove function from here and add it to the __call__
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, function):
def wrapper(*args):
value = function(*args)
# saving to cache codes
return value
return wrapper
@Cache()
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2
Define decorator that takes optional argument:
from functools import wraps, partial
def _cache(func=None, *, instance=None):
if func is None:
return partial(_cache, instance=instance)
@wraps(func)
def wrapper(*ar, **kw):
print(instance)
return func(*ar, **kw)
return wrapper
And pass the instance
object to decorator in __call__
, or use other helper class that is instantiated on each __call__
. This way you can use decorator without brackets, with params or even define a __getattr__
in proxy Cache class to apply some params.
class Cache:
def __call__(self, *ar, **kw):
return _cache(*ar, instance=self, **kw)
cache = Cache()
@cache
def f(): pass
f() # prints <__main__.Cache object at 0x7f5c1bde4880>
You can use a classmethod as a factory method, this should handle all the use cases (with or without parenthesis).
import functools
class Cache():
def __init__(self, function):
functools.update_wrapper(self, function)
self.function = function
self.max_hits = self.__class__.max_hits
self.timeout = self.__class__.timeout
self.cache = {}
def __call__(self, *args):
# Here the code returning the correct thing.
@classmethod
def Cache_dec(cls, _func = None, *, max_hits=10, timeout=5):
cls.max_hits = max_hits
cls.timeout = timeout
if _func is not None: #when decorator is passed parenthesis
return cls(_func)
else:
return cls #when decorator is passed without parenthesis
@Cache.Cache_dec
def double(x):
return x * 2
@Cache.Cache_dec()
def double(x):
return x * 2
@Cache.Cache_dec(timeout=50)
def double(x):
return x * 2
@Cache.Cache_dec(max_hits=100)
def double(x):
return x * 2
@Cache.Cache_dec(max_hits=100, timeout=50)
def double(x):
return x * 2
class myclass2:
def __init__(self,arg):
self.arg=arg
print("call to init")
def __call__(self,func):
print("call to __call__ is made")
self.function=func
def myfunction(x,y,z):
return x+y+z+self.function(x,y,z)
self.newfunction=myfunction
return self.newfunction
@classmethod
def prints(cls,arg):
cls.prints_arg=arg
print("call to prints is made")
return cls(arg)
@myclass2.prints("x")
def myfunction1(x,y,z):
return x+y+z
print(myfunction1(1,2,3))
remember it goes like this:
first call return object get second argument
usually if applicable it goes like argument,function,old function arguments
I made a helper decorator for this purpose:
from functools import update_wrapper
class ClassWrapper:
def __init__(self, cls):
self.cls = cls
def __call__(self, *args, **kwargs):
class ClassWrapperInner:
def __init__(self, cls, *args, **kwargs):
# This combines previous information to get ready to recieve the actual function in the __call__ method.
self._cls = cls
self.args = args
self.kwargs = kwargs
def __call__(self, func, *args, **kw):
# Basically "return self._cls(func, *self.args, **self.kwargs)", but with an adjustment to update the info of the new class & verify correct arguments
assert len(args) == 0 and len(kw) == 0 and callable(func), f"{self._cls.__name__} got invalid arguments. Did you forget to parenthesize?"
obj = self._cls(func, *self.args, **self.kwargs)
update_wrapper(obj, func)
return obj
return ClassWrapperInner(self.cls, *args, **kwargs)
This weird code makes more sense in the context of how it will be executed:
double = ClassWrapper(Cache)(max_hits=100, timeout=50)(double)
ClassWrapper.__init__
stores the class it will be wrapping, (Cache).
ClassWrapper.__call__
passes on its arguments (max_hits=100, timeout=50) to ClassWrapperInner.__init__
, which stores them for the next call.
ClassWrapper.__call__
combines all of the previous arguments and (func) together and gives them to an instance of your class, Cache
, which it returns for use as the new double
. It also updates your class’s arguments, __name__
and __doc__
with the functools library. It’s kind of like a way more complicated version of 2d list flattening where it’s function arguments instead of lists.
With this class decorating it, your original function behaves as expected, except that you need to put parentheses around it in all cases.
@ClassWrapper
class Cache(object):
def __init__(self, function, max_hits=10, timeout=5):
self.function = function
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, *args):
... # Here the code returning the correct thing.
@Cache()
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2
You could try to edit ClassWrapperInner.__call__
so that the parentheses are not required, but this approach is hacky and doesn’t really make sense; it’s like trying to add logic to each method of a class so that calling them without a self parameter works correctly.
EDIT:
After writing this answer, I realized there was a much better way to make the decorator:
def class_wrapper(cls):
def decorator1(*args, **kwargs):
def decorator2(func):
return cls(func, *args, **kwargs)
return decorator2
return decorator1
With functools functions for updating the name & things:
def class_wrapper(cls):
def decorator1(*args, **kwargs):
@wraps(cls)
def decorator2(func):
obj = cls(func, *args, **kwargs)
update_wrapper(obj, func)
return obj
return decorator2
return decorator1
You can also implement the class decorator using a metaclass.
The __call__
method of the metaclass will wrap the original decorator
when the decorator is used with keyword arguments.
class CacheMeta(type):
def __call__(cls, *args, **kwargs):
factory = super().__call__
def wrap(function):
return factory(function, **kwargs)
return wrap if kwargs and not args else wrap(*args)
class Cache(metaclass=CacheMeta):
def __init__(self, function, max_hits=10, timeout=5):
self.function = function
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, *args):
# Here the code returning the correct thing.
@Cache
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2
I’m trying to pass optional arguments to my class decorator in python.
Below the code I currently have:
class Cache(object):
def __init__(self, function, max_hits=10, timeout=5):
self.function = function
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, *args):
# Here the code returning the correct thing.
@Cache
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2
The second decorator with arguments to overwrite the default one (max_hits=10, timeout=5
in my __init__
function), is not working and I got the exception TypeError: __init__() takes at least 2 arguments (3 given)
. I tried many solutions and read articles about it, but here I still can’t make it work.
Any idea to resolve this? Thanks!
@Cache(max_hits=100, timeout=50)
calls __init__(max_hits=100, timeout=50)
, so you aren’t satisfying the function
argument.
You could implement your decorator via a wrapper method that detected whether a function was present. If it finds a function, it can return the Cache object. Otherwise, it can return a wrapper function that will be used as the decorator.
class _Cache(object):
def __init__(self, function, max_hits=10, timeout=5):
self.function = function
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, *args):
# Here the code returning the correct thing.
# wrap _Cache to allow for deferred calling
def Cache(function=None, max_hits=10, timeout=5):
if function:
return _Cache(function)
else:
def wrapper(function):
return _Cache(function, max_hits, timeout)
return wrapper
@Cache
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2
@Cache
def double(...):
...
is equivalent to
def double(...):
...
double=Cache(double)
While
@Cache(max_hits=100, timeout=50)
def double(...):
...
is equivalent to
def double(...):
...
double = Cache(max_hits=100, timeout=50)(double)
Cache(max_hits=100, timeout=50)(double)
has very different semantics than Cache(double)
.
It’s unwise to try to make Cache
handle both use cases.
You could instead use a decorator factory that can take optional max_hits
and timeout
arguments, and returns a decorator:
class Cache(object):
def __init__(self, function, max_hits=10, timeout=5):
self.function = function
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, *args):
# Here the code returning the correct thing.
def cache_hits(max_hits=10, timeout=5):
def _cache(function):
return Cache(function,max_hits,timeout)
return _cache
@cache_hits()
def double(x):
return x * 2
@cache_hits(max_hits=100, timeout=50)
def double(x):
return x * 2
PS. If the class Cache
has no other methods besides __init__
and __call__
, you can probably move all the code inside the _cache
function and eliminate Cache
altogether.
I’ve learned a lot from this question, thanks all. Isn’t the answer just to put empty brackets on the first @Cache
? Then you can move the function
parameter to __call__
.
class Cache(object):
def __init__(self, max_hits=10, timeout=5):
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, function, *args):
# Here the code returning the correct thing.
@Cache()
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2
Although I think this approach is simpler and more concise:
def cache(max_hits=10, timeout=5):
def caching_decorator(fn):
def decorated_fn(*args ,**kwargs):
# Here the code returning the correct thing.
return decorated_fn
return decorator
If you forget the parentheses when using the decorator, unfortunately you still don’t get an error until runtime, as the outer decorator parameters are passed the function you’re trying to decorate. Then at runtime the inner decorator complains:
TypeError: caching_decorator() takes exactly 1 argument (0 given).
However you can catch this, if you know your decorator’s parameters are never going to be a callable:
def cache(max_hits=10, timeout=5):
assert not callable(max_hits), "@cache passed a callable - did you forget to parenthesize?"
def caching_decorator(fn):
def decorated_fn(*args ,**kwargs):
# Here the code returning the correct thing.
return decorated_fn
return decorator
If you now try:
@cache
def some_method()
pass
You get an AssertionError
on declaration.
On a total tangent, I came across this post looking for decorators that decorate classes, rather than classes that decorate. In case anyone else does too, this question is useful.
I’d rather to include the wrapper inside the class’s __call__
method:
UPDATE:
This method has been tested in python 3.6, so I’m not sure about the higher or earlier versions.
class Cache:
def __init__(self, max_hits=10, timeout=5):
# Remove function from here and add it to the __call__
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, function):
def wrapper(*args):
value = function(*args)
# saving to cache codes
return value
return wrapper
@Cache()
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2
Define decorator that takes optional argument:
from functools import wraps, partial
def _cache(func=None, *, instance=None):
if func is None:
return partial(_cache, instance=instance)
@wraps(func)
def wrapper(*ar, **kw):
print(instance)
return func(*ar, **kw)
return wrapper
And pass the instance
object to decorator in __call__
, or use other helper class that is instantiated on each __call__
. This way you can use decorator without brackets, with params or even define a __getattr__
in proxy Cache class to apply some params.
class Cache:
def __call__(self, *ar, **kw):
return _cache(*ar, instance=self, **kw)
cache = Cache()
@cache
def f(): pass
f() # prints <__main__.Cache object at 0x7f5c1bde4880>
You can use a classmethod as a factory method, this should handle all the use cases (with or without parenthesis).
import functools
class Cache():
def __init__(self, function):
functools.update_wrapper(self, function)
self.function = function
self.max_hits = self.__class__.max_hits
self.timeout = self.__class__.timeout
self.cache = {}
def __call__(self, *args):
# Here the code returning the correct thing.
@classmethod
def Cache_dec(cls, _func = None, *, max_hits=10, timeout=5):
cls.max_hits = max_hits
cls.timeout = timeout
if _func is not None: #when decorator is passed parenthesis
return cls(_func)
else:
return cls #when decorator is passed without parenthesis
@Cache.Cache_dec
def double(x):
return x * 2
@Cache.Cache_dec()
def double(x):
return x * 2
@Cache.Cache_dec(timeout=50)
def double(x):
return x * 2
@Cache.Cache_dec(max_hits=100)
def double(x):
return x * 2
@Cache.Cache_dec(max_hits=100, timeout=50)
def double(x):
return x * 2
class myclass2:
def __init__(self,arg):
self.arg=arg
print("call to init")
def __call__(self,func):
print("call to __call__ is made")
self.function=func
def myfunction(x,y,z):
return x+y+z+self.function(x,y,z)
self.newfunction=myfunction
return self.newfunction
@classmethod
def prints(cls,arg):
cls.prints_arg=arg
print("call to prints is made")
return cls(arg)
@myclass2.prints("x")
def myfunction1(x,y,z):
return x+y+z
print(myfunction1(1,2,3))
remember it goes like this:
first call return object get second argument
usually if applicable it goes like argument,function,old function arguments
I made a helper decorator for this purpose:
from functools import update_wrapper
class ClassWrapper:
def __init__(self, cls):
self.cls = cls
def __call__(self, *args, **kwargs):
class ClassWrapperInner:
def __init__(self, cls, *args, **kwargs):
# This combines previous information to get ready to recieve the actual function in the __call__ method.
self._cls = cls
self.args = args
self.kwargs = kwargs
def __call__(self, func, *args, **kw):
# Basically "return self._cls(func, *self.args, **self.kwargs)", but with an adjustment to update the info of the new class & verify correct arguments
assert len(args) == 0 and len(kw) == 0 and callable(func), f"{self._cls.__name__} got invalid arguments. Did you forget to parenthesize?"
obj = self._cls(func, *self.args, **self.kwargs)
update_wrapper(obj, func)
return obj
return ClassWrapperInner(self.cls, *args, **kwargs)
This weird code makes more sense in the context of how it will be executed:
double = ClassWrapper(Cache)(max_hits=100, timeout=50)(double)
ClassWrapper.__init__
stores the class it will be wrapping, (Cache).
ClassWrapper.__call__
passes on its arguments (max_hits=100, timeout=50) to ClassWrapperInner.__init__
, which stores them for the next call.
ClassWrapper.__call__
combines all of the previous arguments and (func) together and gives them to an instance of your class, Cache
, which it returns for use as the new double
. It also updates your class’s arguments, __name__
and __doc__
with the functools library. It’s kind of like a way more complicated version of 2d list flattening where it’s function arguments instead of lists.
With this class decorating it, your original function behaves as expected, except that you need to put parentheses around it in all cases.
@ClassWrapper
class Cache(object):
def __init__(self, function, max_hits=10, timeout=5):
self.function = function
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, *args):
... # Here the code returning the correct thing.
@Cache()
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2
You could try to edit ClassWrapperInner.__call__
so that the parentheses are not required, but this approach is hacky and doesn’t really make sense; it’s like trying to add logic to each method of a class so that calling them without a self parameter works correctly.
EDIT:
After writing this answer, I realized there was a much better way to make the decorator:
def class_wrapper(cls):
def decorator1(*args, **kwargs):
def decorator2(func):
return cls(func, *args, **kwargs)
return decorator2
return decorator1
With functools functions for updating the name & things:
def class_wrapper(cls):
def decorator1(*args, **kwargs):
@wraps(cls)
def decorator2(func):
obj = cls(func, *args, **kwargs)
update_wrapper(obj, func)
return obj
return decorator2
return decorator1
You can also implement the class decorator using a metaclass.
The __call__
method of the metaclass will wrap the original decorator
when the decorator is used with keyword arguments.
class CacheMeta(type):
def __call__(cls, *args, **kwargs):
factory = super().__call__
def wrap(function):
return factory(function, **kwargs)
return wrap if kwargs and not args else wrap(*args)
class Cache(metaclass=CacheMeta):
def __init__(self, function, max_hits=10, timeout=5):
self.function = function
self.max_hits = max_hits
self.timeout = timeout
self.cache = {}
def __call__(self, *args):
# Here the code returning the correct thing.
@Cache
def double(x):
return x * 2
@Cache(max_hits=100, timeout=50)
def double(x):
return x * 2