Python decorator best practice, using a class vs a function
Question:
As I’ve understood it there are two ways to do a Python decorator, to either use the __call__ of a class or to define and call a function as the decorator. What’s the advantages/disadvantages of these methods? Is there one preferred method?
Example 1
class dec1(object):
def __init__(self, f):
self.f = f
def __call__(self):
print "Decorating", self.f.__name__
self.f()
@dec1
def func1():
print "inside func1()"
func1()
# Decorating func1
# inside func1()
Example 2
def dec2(f):
def new_f():
print "Decorating", f.__name__
f()
return new_f
@dec2
def func2():
print "inside func2()"
func2()
# Decorating func2
# inside func2()
Answers:
There are two different decorator implementations. One of these uses a class as a decorator and the other uses a function as a decorator. You must choose the preferred implementation for your needs.
For example, if your decorator does a lot of work then you can use class as a decorator, like this:
import logging
import time
import pymongo
import hashlib
import random
DEBUG_MODE = True
class logger(object):
def __new__(cls, *args, **kwargs):
if DEBUG_MODE:
return object.__new__(cls, *args, **kwargs)
else:
return args[0]
def __init__(self, foo):
self.foo = foo
logging.basicConfig(filename='exceptions.log', format='%(levelname)s % (asctime)s: %(message)s')
self.log = logging.getLogger(__name__)
def __call__(self, *args, **kwargs):
def _log():
try:
t = time.time()
func_hash = self._make_hash(t)
col = self._make_db_connection()
log_record = {'func_name':self.foo.__name__, 'start_time':t, 'func_hash':func_hash}
col.insert(log_record)
res = self.foo(*args, **kwargs)
log_record = {'func_name':self.foo.__name__, 'exc_time':round(time.time() - t,4), 'end_time':time.time(),'func_hash':func_hash}
col.insert(log_record)
return res
except Exception as e:
self.log.error(e)
return _log()
def _make_db_connection(self):
connection = pymongo.Connection()
db = connection.logger
collection = db.log
return collection
def _make_hash(self, t):
m = hashlib.md5()
m.update(str(t)+str(random.randrange(1,10)))
return m.hexdigest()
It is rather subjective to say whether there are "advantages" to each method.
However, a good understanding of what goes under the hood would make it natural
for one to pick the best choice for each occasion.
A decorator (talking about function decorators), is simply a callable object that takes a function as its input parameter. Python has its rather interesting design that allows
one to create other kinds of callable objects, besides functions – and one can put that to use
to create more maintainable or shorter code on occasion.
Decorators were added back in Python 2.3 as a "syntactic shortcut" for
def a(x):
...
a = my_decorator(a)
Besides that, we usually call decorators some "callables" that would rather be "decorator factories" – when we use this kind:
@my_decorator(param1, param2)
def my_func(...):
...
the call is made to "my_decorator" with param1 and param2 – it then returns an object that will be called again, this time having "my_func" as a parameter. So, in this case, technically the "decorator" is whatever is returned by the "my_decorator", making it a
"decorator factory".
Now, either decorators or "decorator factories" as described usually have to keep some internal state. In the first case, the only thing it does keep is a reference to the original function (the variable called f in your examples). A "decorator factory" may want to register extra state variables ("param1" and "param2" in the example above).
This extra state, in the case of decorators written as functions is kept in variables within the enclosing functions, and accessed as "nonlocal" variables by the actual wrapper function. If one writes a proper class, they can be kept as instance variables in the decorator function (which will be seen as a "callable object", not a "function") – and access to them is more explicit and more readable.
So, for most cases it is a matter of readability whether you will prefer one approach or the other:
for short, simple decorators, the functional approach is often more readable than one written as a class – while sometimes a more elaborate one – especially one "decorator factory" will take full advantage of the "flat is better than nested" advice fore Python coding.
Consider:
def my_dec_factory(param1, param2):
...
...
def real_decorator(func):
...
def wraper_func(*args, **kwargs):
...
#use param1
result = func(*args, **kwargs)
#use param2
return result
return wraper_func
return real_decorator
against this "hybrid" solution:
class MyDecorator(object):
"""Decorator example mixing class and function definitions."""
def __init__(self, func, param1, param2):
self.func = func
self.param1, self.param2 = param1, param2
def __call__(self, *args, **kwargs):
...
#use self.param1
result = self.func(*args, **kwargs)
#use self.param2
return result
def my_dec_factory(param1, param2):
def decorator(func):
return MyDecorator(func, param1, param2)
return decorator
update: Missing "pure class" forms of decorators
Now, note the "hybrid" method takes the "best of both Worlds" trying to keep the shortest and more readable code. A full "decorator factory" defined exclusively with classes would either need two classes, or a "mode" attribute to know if it was called to register the decorated function or to actually call the final function:
class MyDecorator(object):
"""Decorator example defined entirely as class."""
def __init__(self, p1, p2):
self.p1 = p1
...
self.mode = "decorating"
def __call__(self, *args, **kw):
if self.mode == "decorating":
self.func = args[0]
self.mode = "calling"
return self
# code to run prior to function call
result = self.func(*args, **kw)
# code to run after function call
return result
@MyDecorator(p1, ...)
def myfunc():
...
And finally a pure, "white colar" decorator defined with two classes – maybe keeping things more separated, but increasing the redundancy to a point one can’t say it is more maintainable:
class Stage2Decorator(object):
def __init__(self, func, p1, p2, ...):
self.func = func
self.p1 = p1
...
def __call__(self, *args, **kw):
# code to run prior to function call
...
result = self.func(*args, **kw)
# code to run after function call
...
return result
class Stage1Decorator(object):
"""Decorator example defined as two classes.
No "hacks" on the object model, most bureacratic.
"""
def __init__(self, p1, p2):
self.p1 = p1
...
self.mode = "decorating"
def __call__(self, func):
return Stage2Decorator(func, self.p1, self.p2, ...)
@Stage1Decorator(p1, p2, ...)
def myfunc():
...
2018 update
I wrote the text above a couple years ago. I came up recently with a pattern I prefer due to creating code that is "flatter".
The basic idea is to use a function, but return a partial object of itself if it is called with parameters before being used as a decorator:
from functools import wraps, partial
def decorator(func=None, parameter1=None, parameter2=None, ...):
if not func:
# The only drawback is that for functions there is no thing
# like "self" - we have to rely on the decorator
# function name on the module namespace
return partial(decorator, parameter1=parameter1, parameter2=parameter2)
@wraps(func)
def wrapper(*args, **kwargs):
# Decorator code- parameter1, etc... can be used
# freely here
return func(*args, **kwargs)
return wrapper
And that is it – decorators written using this pattern can decorate
a function right away without being "called" first:
@decorator
def my_func():
pass
Or customized with parameters:
@decorator(parameter1="example.com", ...):
def my_func():
pass
2019 – With Python 3.8 and positional only parameters this last pattern will become even better, as the func argument can be declared as positional only, and require the parameters to be named;
def decorator(func=None, /, *, parameter1=None, parameter2=None, ...):
I mostly agree with jsbueno: there’s no one right way. It depends on the situation. But I think def is probably better in most cases, because if you go with class, most of the “real” work is going to be done in __call__ anyway. Also, callables that are not functions are pretty rare (with the notable exception of instantiating a class), and people generally do not expect that. Also, local variables are usually easier for people to keep track of vs. instance variables, simply because they have more limited scope, although in this case, the instance variables are probably only used in __call__ (with __init__ simply copying them from arguments).
I have to disagree with his hybrid approach though. It’s an interesting design, but I think it’s probably going to confuse the crap out of you or someone else who looks at it a few months later.
Tangent: Regardless of whether you go with class or function, you should use functools.wraps, which itself is meant to be used as a decorator (we must go deeper!) like so:
import functools
def require_authorization(f):
@functools.wraps(f)
def decorated(user, *args, **kwargs):
if not is_authorized(user):
raise UserIsNotAuthorized
return f(user, *args, **kwargs)
return decorated
@require_authorization
def check_email(user, etc):
# etc.
This makes decorated look like check_email e.g. by changing it’s func_name attribute.
Anyway, this is usually what I do and what I see other people around me doing, unless I want a decorator factory. In that case, I just add another level of def:
def require_authorization(action):
def decorate(f):
@functools.wraps(f):
def decorated(user, *args, **kwargs):
if not is_allowed_to(user, action):
raise UserIsNotAuthorized(action, user)
return f(user, *args, **kwargs)
return decorated
return decorate
By the way, I would also be on guard against excessive use of decorators, because they can make it really hard to follow stack traces.
One approach for managing hideous stack traces is to have a policy of not substantially changing the behavior of the decoratee. E.g.
def log_call(f):
@functools.wraps(f)
def decorated(*args, **kwargs):
logging.debug('call being made: %s(*%r, **%r)',
f.func_name, args, kwargs)
return f(*args, **kwargs)
return decorated
A more extreme approach for keeping your stack traces sane is for the decorator to return the decoratee unmodified, like so:
import threading
DEPRECATED_LOCK = threading.Lock()
DEPRECATED = set()
def deprecated(f):
with DEPRECATED_LOCK:
DEPRECATED.add(f)
return f
@deprecated
def old_hack():
# etc.
This is useful if the function is called within a framework that knows about the deprecated decorator. E.g.
class MyLamerFramework(object):
def register_handler(self, maybe_deprecated):
if not self.allow_deprecated and is_deprecated(f):
raise ValueError(
'Attempted to register deprecated function %s as a handler.'
% f.func_name)
self._handlers.add(maybe_deprecated)
I will dare to offer a different approach to the problem almost seven years after the question was originally made. This version is not described in any of the previous (very nice!) answers.
The biggest differences between using classes and functions as decorators are already very well described here. For the sake of completeness I’ll go briefly through this again, but to be more practical, I’m going to use a concrete example.
Let’s say you want to write a decorator to cache the result of “pure” functions (those free of side effects, so the return value is deterministic, given the arguments) in some cache service.
Here are two equivalent and very simple decorators for doing this, in both flavors (functional and object oriented):
import json
import your_cache_service as cache
def cache_func(f):
def wrapper(*args, **kwargs):
key = json.dumps([f.__name__, args, kwargs])
cached_value = cache.get(key)
if cached_value is not None:
print('cache HIT')
return cached_value
print('cache MISS')
value = f(*args, **kwargs)
cache.set(key, value)
return value
return wrapper
class CacheClass(object):
def __init__(self, f):
self.orig_func = f
def __call__(self, *args, **kwargs):
key = json.dumps([self.orig_func.__name__, args, kwargs])
cached_value = cache.get(key)
if cached_value is not None:
print('cache HIT')
return cached_value
print('cache MISS')
value = self.orig_func(*args, **kwargs)
cache.set(key, value)
return value
I guess this is fairly easy to understand. It’s just a silly example! I’m skipping all error handling and edge cases for simplicity. You should not ctrl+c/ctrl+v code from StackOverflow anyways, right? 😉
As one can notice, both versions are essentially the same. The object oriented version is a bit longer and more verbose than the functional one, because we have to define methods and use the variable self, but I would argue it is slightly more readable. This factor becomes really important for more complex decorators. We’ll see that in a moment.
The decorators above are used like this:
@cache_func
def test_one(a, b=0, c=1):
return (a + b)*c
# Behind the scenes:
# test_one = cache_func(test_one)
print(test_one(3, 4, 6))
print(test_one(3, 4, 6))
# Prints:
# cache MISS
# 42
# cache HIT
# 42
@CacheClass
def test_two(x, y=0, z=1):
return (x + y)*z
# Behind the scenes:
# test_two = CacheClass(test_two)
print(test_two(1, 1, 569))
print(test_two(1, 1, 569))
# Prints:
# cache MISS
# 1138
# cache HIT
# 1138
But let’s say now that your cache service supports setting the TTL for each cache entry. You would need to defined that on decoration time. How to do it?
The traditional functional approach would be to add a new wrapper layer that returns a configured decorator (there are nicer suggestions in the other answers to this question):
import json
import your_cache_service as cache
def cache_func_with_options(ttl=None):
def configured_decorator(*args, **kwargs):
def wrapper(*args, **kwargs):
key = json.dumps([f.__name__, args, kwargs])
cached_value = cache.get(key)
if cached_value is not None:
print('cache HIT')
return cached_value
print('cache MISS')
value = f(*args, **kwargs)
cache.set(key, value, ttl=ttl)
return value
return wrapper
return configured_decorator
It is used like this:
from time import sleep
@cache_func_with_options(ttl=100)
def test_three(a, b=0, c=1):
return hex((a + b)*c)
# Behind the scenes:
# test_three = cache_func_with_options(ttl=100)(test_three)
print(test_three(8731))
print(test_three(8731))
sleep(0.2)
print(test_three(8731))
# Prints:
# cache MISS
# 0x221b
# cache HIT
# 0x221b
# cache MISS
# 0x221b
This one is still okay, but I have to admit that, even being an experienced developer, sometimes I see myself taking a good amount of time to understand more complex decorators that follow this pattern. The tricky part here is that is really not possible to “un-nest” the functions, as the inner functions need the variables defined in the scope of the outer ones.
Can the object oriented version help? I think so, but if you follow the previous structure for the class-based one, it would end up with the same nested structure as the functional one or, even worse, using flags to hold the state of what the decorator is doing (not nice).
So, instead of receiving the function to be decorated in the __init__ method and handling the wrapping and decorator parameters in the __call__ method (or using multiple classes/functions to do so, which is too complex to my taste), my suggestion is to handle the decorator parameters in the __init__ method, receive the function in the __call__ method and finally handle the wrapping in an additional method that is returned by the end of the __call__.
It looks like this:
import json
import your_cache_service as cache
class CacheClassWithOptions(object):
def __init__(self, ttl=None):
self.ttl = ttl
def __call__(self, f):
self.orig_func = f
return self.wrapper
def wrapper(self, *args, **kwargs):
key = json.dumps([self.orig_func.__name__, args, kwargs])
cached_value = cache.get(key)
if cached_value is not None:
print('cache HIT')
return cached_value
print('cache MISS')
value = self.orig_func(*args, **kwargs)
cache.set(key, value, ttl=self.ttl)
return value
The usage is as expected:
from time import sleep
@CacheClassWithOptions(ttl=100)
def test_four(x, y=0, z=1):
return (x + y)*z
# Behind the scenes:
# test_four = CacheClassWithOptions(ttl=100)(test_four)
print(test_four(21, 42, 27))
print(test_four(21, 42, 27))
sleep(0.2)
print(test_four(21, 42, 27))
# Prints:
# cache MISS
# 1701
# cache HIT
# 1701
# cache MISS
# 1701
As anything is perfect, there are two small drawbacks with this last approach:
-
It’s not possible to decorate using @CacheClassWithOptions directly. We have to use parenthesis @CacheClassWithOptions(), even if we don’t want to pass any parameter. This is because we need to create the instance first, before trying to decorate, so the __call__ method will receive the function to be decorated, not in the __init__. It is possible to work around this limitation, but it’s very hacky. Better to simply accept that those parenthesis are needed.
-
There’s no obvious place to apply the functools.wraps decorator on the returned wrapped function, what would be a no-brainer in the functional version. It can easily be done, though, by creating an intermediary function inside __call__ before returning. It just doesn’t look that nice and it’s better to leave that out if you don’t need the nice things that functools.wraps does.
As I’ve understood it there are two ways to do a Python decorator, to either use the __call__ of a class or to define and call a function as the decorator. What’s the advantages/disadvantages of these methods? Is there one preferred method?
Example 1
class dec1(object):
def __init__(self, f):
self.f = f
def __call__(self):
print "Decorating", self.f.__name__
self.f()
@dec1
def func1():
print "inside func1()"
func1()
# Decorating func1
# inside func1()
Example 2
def dec2(f):
def new_f():
print "Decorating", f.__name__
f()
return new_f
@dec2
def func2():
print "inside func2()"
func2()
# Decorating func2
# inside func2()
There are two different decorator implementations. One of these uses a class as a decorator and the other uses a function as a decorator. You must choose the preferred implementation for your needs.
For example, if your decorator does a lot of work then you can use class as a decorator, like this:
import logging
import time
import pymongo
import hashlib
import random
DEBUG_MODE = True
class logger(object):
def __new__(cls, *args, **kwargs):
if DEBUG_MODE:
return object.__new__(cls, *args, **kwargs)
else:
return args[0]
def __init__(self, foo):
self.foo = foo
logging.basicConfig(filename='exceptions.log', format='%(levelname)s % (asctime)s: %(message)s')
self.log = logging.getLogger(__name__)
def __call__(self, *args, **kwargs):
def _log():
try:
t = time.time()
func_hash = self._make_hash(t)
col = self._make_db_connection()
log_record = {'func_name':self.foo.__name__, 'start_time':t, 'func_hash':func_hash}
col.insert(log_record)
res = self.foo(*args, **kwargs)
log_record = {'func_name':self.foo.__name__, 'exc_time':round(time.time() - t,4), 'end_time':time.time(),'func_hash':func_hash}
col.insert(log_record)
return res
except Exception as e:
self.log.error(e)
return _log()
def _make_db_connection(self):
connection = pymongo.Connection()
db = connection.logger
collection = db.log
return collection
def _make_hash(self, t):
m = hashlib.md5()
m.update(str(t)+str(random.randrange(1,10)))
return m.hexdigest()
It is rather subjective to say whether there are "advantages" to each method.
However, a good understanding of what goes under the hood would make it natural
for one to pick the best choice for each occasion.
A decorator (talking about function decorators), is simply a callable object that takes a function as its input parameter. Python has its rather interesting design that allows
one to create other kinds of callable objects, besides functions – and one can put that to use
to create more maintainable or shorter code on occasion.
Decorators were added back in Python 2.3 as a "syntactic shortcut" for
def a(x):
...
a = my_decorator(a)
Besides that, we usually call decorators some "callables" that would rather be "decorator factories" – when we use this kind:
@my_decorator(param1, param2)
def my_func(...):
...
the call is made to "my_decorator" with param1 and param2 – it then returns an object that will be called again, this time having "my_func" as a parameter. So, in this case, technically the "decorator" is whatever is returned by the "my_decorator", making it a
"decorator factory".
Now, either decorators or "decorator factories" as described usually have to keep some internal state. In the first case, the only thing it does keep is a reference to the original function (the variable called f in your examples). A "decorator factory" may want to register extra state variables ("param1" and "param2" in the example above).
This extra state, in the case of decorators written as functions is kept in variables within the enclosing functions, and accessed as "nonlocal" variables by the actual wrapper function. If one writes a proper class, they can be kept as instance variables in the decorator function (which will be seen as a "callable object", not a "function") – and access to them is more explicit and more readable.
So, for most cases it is a matter of readability whether you will prefer one approach or the other:
for short, simple decorators, the functional approach is often more readable than one written as a class – while sometimes a more elaborate one – especially one "decorator factory" will take full advantage of the "flat is better than nested" advice fore Python coding.
Consider:
def my_dec_factory(param1, param2):
...
...
def real_decorator(func):
...
def wraper_func(*args, **kwargs):
...
#use param1
result = func(*args, **kwargs)
#use param2
return result
return wraper_func
return real_decorator
against this "hybrid" solution:
class MyDecorator(object):
"""Decorator example mixing class and function definitions."""
def __init__(self, func, param1, param2):
self.func = func
self.param1, self.param2 = param1, param2
def __call__(self, *args, **kwargs):
...
#use self.param1
result = self.func(*args, **kwargs)
#use self.param2
return result
def my_dec_factory(param1, param2):
def decorator(func):
return MyDecorator(func, param1, param2)
return decorator
update: Missing "pure class" forms of decorators
Now, note the "hybrid" method takes the "best of both Worlds" trying to keep the shortest and more readable code. A full "decorator factory" defined exclusively with classes would either need two classes, or a "mode" attribute to know if it was called to register the decorated function or to actually call the final function:
class MyDecorator(object):
"""Decorator example defined entirely as class."""
def __init__(self, p1, p2):
self.p1 = p1
...
self.mode = "decorating"
def __call__(self, *args, **kw):
if self.mode == "decorating":
self.func = args[0]
self.mode = "calling"
return self
# code to run prior to function call
result = self.func(*args, **kw)
# code to run after function call
return result
@MyDecorator(p1, ...)
def myfunc():
...
And finally a pure, "white colar" decorator defined with two classes – maybe keeping things more separated, but increasing the redundancy to a point one can’t say it is more maintainable:
class Stage2Decorator(object):
def __init__(self, func, p1, p2, ...):
self.func = func
self.p1 = p1
...
def __call__(self, *args, **kw):
# code to run prior to function call
...
result = self.func(*args, **kw)
# code to run after function call
...
return result
class Stage1Decorator(object):
"""Decorator example defined as two classes.
No "hacks" on the object model, most bureacratic.
"""
def __init__(self, p1, p2):
self.p1 = p1
...
self.mode = "decorating"
def __call__(self, func):
return Stage2Decorator(func, self.p1, self.p2, ...)
@Stage1Decorator(p1, p2, ...)
def myfunc():
...
2018 update
I wrote the text above a couple years ago. I came up recently with a pattern I prefer due to creating code that is "flatter".
The basic idea is to use a function, but return a partial object of itself if it is called with parameters before being used as a decorator:
from functools import wraps, partial
def decorator(func=None, parameter1=None, parameter2=None, ...):
if not func:
# The only drawback is that for functions there is no thing
# like "self" - we have to rely on the decorator
# function name on the module namespace
return partial(decorator, parameter1=parameter1, parameter2=parameter2)
@wraps(func)
def wrapper(*args, **kwargs):
# Decorator code- parameter1, etc... can be used
# freely here
return func(*args, **kwargs)
return wrapper
And that is it – decorators written using this pattern can decorate
a function right away without being "called" first:
@decorator
def my_func():
pass
Or customized with parameters:
@decorator(parameter1="example.com", ...):
def my_func():
pass
2019 – With Python 3.8 and positional only parameters this last pattern will become even better, as the func argument can be declared as positional only, and require the parameters to be named;
def decorator(func=None, /, *, parameter1=None, parameter2=None, ...):
I mostly agree with jsbueno: there’s no one right way. It depends on the situation. But I think def is probably better in most cases, because if you go with class, most of the “real” work is going to be done in __call__ anyway. Also, callables that are not functions are pretty rare (with the notable exception of instantiating a class), and people generally do not expect that. Also, local variables are usually easier for people to keep track of vs. instance variables, simply because they have more limited scope, although in this case, the instance variables are probably only used in __call__ (with __init__ simply copying them from arguments).
I have to disagree with his hybrid approach though. It’s an interesting design, but I think it’s probably going to confuse the crap out of you or someone else who looks at it a few months later.
Tangent: Regardless of whether you go with class or function, you should use functools.wraps, which itself is meant to be used as a decorator (we must go deeper!) like so:
import functools
def require_authorization(f):
@functools.wraps(f)
def decorated(user, *args, **kwargs):
if not is_authorized(user):
raise UserIsNotAuthorized
return f(user, *args, **kwargs)
return decorated
@require_authorization
def check_email(user, etc):
# etc.
This makes decorated look like check_email e.g. by changing it’s func_name attribute.
Anyway, this is usually what I do and what I see other people around me doing, unless I want a decorator factory. In that case, I just add another level of def:
def require_authorization(action):
def decorate(f):
@functools.wraps(f):
def decorated(user, *args, **kwargs):
if not is_allowed_to(user, action):
raise UserIsNotAuthorized(action, user)
return f(user, *args, **kwargs)
return decorated
return decorate
By the way, I would also be on guard against excessive use of decorators, because they can make it really hard to follow stack traces.
One approach for managing hideous stack traces is to have a policy of not substantially changing the behavior of the decoratee. E.g.
def log_call(f):
@functools.wraps(f)
def decorated(*args, **kwargs):
logging.debug('call being made: %s(*%r, **%r)',
f.func_name, args, kwargs)
return f(*args, **kwargs)
return decorated
A more extreme approach for keeping your stack traces sane is for the decorator to return the decoratee unmodified, like so:
import threading
DEPRECATED_LOCK = threading.Lock()
DEPRECATED = set()
def deprecated(f):
with DEPRECATED_LOCK:
DEPRECATED.add(f)
return f
@deprecated
def old_hack():
# etc.
This is useful if the function is called within a framework that knows about the deprecated decorator. E.g.
class MyLamerFramework(object):
def register_handler(self, maybe_deprecated):
if not self.allow_deprecated and is_deprecated(f):
raise ValueError(
'Attempted to register deprecated function %s as a handler.'
% f.func_name)
self._handlers.add(maybe_deprecated)
I will dare to offer a different approach to the problem almost seven years after the question was originally made. This version is not described in any of the previous (very nice!) answers.
The biggest differences between using classes and functions as decorators are already very well described here. For the sake of completeness I’ll go briefly through this again, but to be more practical, I’m going to use a concrete example.
Let’s say you want to write a decorator to cache the result of “pure” functions (those free of side effects, so the return value is deterministic, given the arguments) in some cache service.
Here are two equivalent and very simple decorators for doing this, in both flavors (functional and object oriented):
import json
import your_cache_service as cache
def cache_func(f):
def wrapper(*args, **kwargs):
key = json.dumps([f.__name__, args, kwargs])
cached_value = cache.get(key)
if cached_value is not None:
print('cache HIT')
return cached_value
print('cache MISS')
value = f(*args, **kwargs)
cache.set(key, value)
return value
return wrapper
class CacheClass(object):
def __init__(self, f):
self.orig_func = f
def __call__(self, *args, **kwargs):
key = json.dumps([self.orig_func.__name__, args, kwargs])
cached_value = cache.get(key)
if cached_value is not None:
print('cache HIT')
return cached_value
print('cache MISS')
value = self.orig_func(*args, **kwargs)
cache.set(key, value)
return value
I guess this is fairly easy to understand. It’s just a silly example! I’m skipping all error handling and edge cases for simplicity. You should not ctrl+c/ctrl+v code from StackOverflow anyways, right? 😉
As one can notice, both versions are essentially the same. The object oriented version is a bit longer and more verbose than the functional one, because we have to define methods and use the variable self, but I would argue it is slightly more readable. This factor becomes really important for more complex decorators. We’ll see that in a moment.
The decorators above are used like this:
@cache_func
def test_one(a, b=0, c=1):
return (a + b)*c
# Behind the scenes:
# test_one = cache_func(test_one)
print(test_one(3, 4, 6))
print(test_one(3, 4, 6))
# Prints:
# cache MISS
# 42
# cache HIT
# 42
@CacheClass
def test_two(x, y=0, z=1):
return (x + y)*z
# Behind the scenes:
# test_two = CacheClass(test_two)
print(test_two(1, 1, 569))
print(test_two(1, 1, 569))
# Prints:
# cache MISS
# 1138
# cache HIT
# 1138
But let’s say now that your cache service supports setting the TTL for each cache entry. You would need to defined that on decoration time. How to do it?
The traditional functional approach would be to add a new wrapper layer that returns a configured decorator (there are nicer suggestions in the other answers to this question):
import json
import your_cache_service as cache
def cache_func_with_options(ttl=None):
def configured_decorator(*args, **kwargs):
def wrapper(*args, **kwargs):
key = json.dumps([f.__name__, args, kwargs])
cached_value = cache.get(key)
if cached_value is not None:
print('cache HIT')
return cached_value
print('cache MISS')
value = f(*args, **kwargs)
cache.set(key, value, ttl=ttl)
return value
return wrapper
return configured_decorator
It is used like this:
from time import sleep
@cache_func_with_options(ttl=100)
def test_three(a, b=0, c=1):
return hex((a + b)*c)
# Behind the scenes:
# test_three = cache_func_with_options(ttl=100)(test_three)
print(test_three(8731))
print(test_three(8731))
sleep(0.2)
print(test_three(8731))
# Prints:
# cache MISS
# 0x221b
# cache HIT
# 0x221b
# cache MISS
# 0x221b
This one is still okay, but I have to admit that, even being an experienced developer, sometimes I see myself taking a good amount of time to understand more complex decorators that follow this pattern. The tricky part here is that is really not possible to “un-nest” the functions, as the inner functions need the variables defined in the scope of the outer ones.
Can the object oriented version help? I think so, but if you follow the previous structure for the class-based one, it would end up with the same nested structure as the functional one or, even worse, using flags to hold the state of what the decorator is doing (not nice).
So, instead of receiving the function to be decorated in the __init__ method and handling the wrapping and decorator parameters in the __call__ method (or using multiple classes/functions to do so, which is too complex to my taste), my suggestion is to handle the decorator parameters in the __init__ method, receive the function in the __call__ method and finally handle the wrapping in an additional method that is returned by the end of the __call__.
It looks like this:
import json
import your_cache_service as cache
class CacheClassWithOptions(object):
def __init__(self, ttl=None):
self.ttl = ttl
def __call__(self, f):
self.orig_func = f
return self.wrapper
def wrapper(self, *args, **kwargs):
key = json.dumps([self.orig_func.__name__, args, kwargs])
cached_value = cache.get(key)
if cached_value is not None:
print('cache HIT')
return cached_value
print('cache MISS')
value = self.orig_func(*args, **kwargs)
cache.set(key, value, ttl=self.ttl)
return value
The usage is as expected:
from time import sleep
@CacheClassWithOptions(ttl=100)
def test_four(x, y=0, z=1):
return (x + y)*z
# Behind the scenes:
# test_four = CacheClassWithOptions(ttl=100)(test_four)
print(test_four(21, 42, 27))
print(test_four(21, 42, 27))
sleep(0.2)
print(test_four(21, 42, 27))
# Prints:
# cache MISS
# 1701
# cache HIT
# 1701
# cache MISS
# 1701
As anything is perfect, there are two small drawbacks with this last approach:
-
It’s not possible to decorate using
@CacheClassWithOptionsdirectly. We have to use parenthesis@CacheClassWithOptions(), even if we don’t want to pass any parameter. This is because we need to create the instance first, before trying to decorate, so the__call__method will receive the function to be decorated, not in the__init__. It is possible to work around this limitation, but it’s very hacky. Better to simply accept that those parenthesis are needed. -
There’s no obvious place to apply the
functools.wrapsdecorator on the returned wrapped function, what would be a no-brainer in the functional version. It can easily be done, though, by creating an intermediary function inside__call__before returning. It just doesn’t look that nice and it’s better to leave that out if you don’t need the nice things thatfunctools.wrapsdoes.