Simulating a 'local static' variable in python
Question:
Consider the following code:
def CalcSomething(a):
if CalcSomething._cache.has_key(a):
return CalcSomething._cache[a]
CalcSomething._cache[a] = ReallyCalc(a)
return CalcSomething._cache[a]
CalcSomething._cache = { }
This is the easiest way I can think of for simulating a ‘local static’ variable in python.
What bothers me is that CalcSomething._cache is mentioned outside the function’s definition, but the alternative would be something like that:
if not hasattr(CalcSomething, "_cache"):
setattr(CalcSomething, "_cache", { } )
inside the function’s definition, which is really cumbersome.
Is there a more elegant way?
[EDIT]
Just to clarify, this question is not about local function caches, as the example above might suggest. Here is another short example where a ‘static local’ might be handy:
def ParseString(s):
return ParseString._parser.parse(s)
# Create a Parser object once, which will be used for all parsings.
# Assuming a Parser object is heave on resources, for the sake of this example.
ParseString._parser = Parser()
Answers:
Turn it into a decorator.
def static_var(var_name, initial_value):
def _set_var(obj):
setattr(obj, var_name, initial_value)
return obj
return _set_var
@static_var("_cache", {})
def CalcSomething(a):
...
Consider writing decorator that will maintain cache and your function won’t be contaminated by caching code:
def cacheResults(aFunc):
'''This decorator funcion binds a map between the tuple of arguments
and results computed by aFunc for those arguments'''
def cachedFunc(*args):
if not hasattr(aFunc, '_cache'):
aFunc._cache = {}
if args in aFunc._cache:
return aFunc._cache[args]
newVal = aFunc(*args)
aFunc._cache[args] = newVal
return newVal
return cachedFunc
@cacheResults
def ReallyCalc(a):
'''This function does only actual computation'''
return pow(a, 42)
Maybe it doesn’t look great at first, but you can use cacheResults()
anywhere you don’t need keyword parameters. It is possible to create similar decorator that would work also for keyword params, but that didn’t seem necessary this time.
Turn it into a callable object (since that’s what it really is.)
class CalcSomething(object):
def __init__(self):
self._cache = {}
def __call__(self, a):
if a not in self._cache:
self._cache[a] = self.reallyCalc(a)
return self._cache[a]
def reallyCalc(self, a):
return # a real answer
calcSomething = CalcSomething()
Now you can use calcSomething
as if it were a function. But it remains tidy and self-contained.
One option is to abuse default parameters. ie:
def CalcSomething(a, _cache={}):
if _cache.has_key(a):
This has the advantage that you don’t need to qualify the name, and will get fast local access to the variables rather than doing two slow dict lookups. However it still has the problem that it is mentioned outside the function (in fact it’s worse since its now in the function signature.)
To prevent this, a better solution would be to wrap the function in a closure containing your statics:
@apply
def CalcSomething():
cache = {} # statics go here
def CalcSomething(a):
if cache.has_key(a):
return cache[a]
cache[a] = ReallyCalc(a)
return cache[a]
return CalcSomething
The solution proposed by S.Lott is the solution I would propose too.
There are useful “memoize” decorators around, too, like:
- Memoize decorator function with cache size limit
- Memoize decorator with O(1) length-limited LRU cache, supports mutable types
Given all that, I’m providing an alternative for your initial attempt at a function and a “static local”, which is standalone:
def calc_something(a):
try:
return calc_something._cache[a]
except AttributeError: # _cache is not there
calc_something._cache= {}
except KeyError: # the result is not there
pass
# compute result here
calc_something._cache[a]= result
return result
Consider the following code:
def CalcSomething(a):
if CalcSomething._cache.has_key(a):
return CalcSomething._cache[a]
CalcSomething._cache[a] = ReallyCalc(a)
return CalcSomething._cache[a]
CalcSomething._cache = { }
This is the easiest way I can think of for simulating a ‘local static’ variable in python.
What bothers me is that CalcSomething._cache is mentioned outside the function’s definition, but the alternative would be something like that:
if not hasattr(CalcSomething, "_cache"):
setattr(CalcSomething, "_cache", { } )
inside the function’s definition, which is really cumbersome.
Is there a more elegant way?
[EDIT]
Just to clarify, this question is not about local function caches, as the example above might suggest. Here is another short example where a ‘static local’ might be handy:
def ParseString(s):
return ParseString._parser.parse(s)
# Create a Parser object once, which will be used for all parsings.
# Assuming a Parser object is heave on resources, for the sake of this example.
ParseString._parser = Parser()
Turn it into a decorator.
def static_var(var_name, initial_value):
def _set_var(obj):
setattr(obj, var_name, initial_value)
return obj
return _set_var
@static_var("_cache", {})
def CalcSomething(a):
...
Consider writing decorator that will maintain cache and your function won’t be contaminated by caching code:
def cacheResults(aFunc):
'''This decorator funcion binds a map between the tuple of arguments
and results computed by aFunc for those arguments'''
def cachedFunc(*args):
if not hasattr(aFunc, '_cache'):
aFunc._cache = {}
if args in aFunc._cache:
return aFunc._cache[args]
newVal = aFunc(*args)
aFunc._cache[args] = newVal
return newVal
return cachedFunc
@cacheResults
def ReallyCalc(a):
'''This function does only actual computation'''
return pow(a, 42)
Maybe it doesn’t look great at first, but you can use cacheResults()
anywhere you don’t need keyword parameters. It is possible to create similar decorator that would work also for keyword params, but that didn’t seem necessary this time.
Turn it into a callable object (since that’s what it really is.)
class CalcSomething(object):
def __init__(self):
self._cache = {}
def __call__(self, a):
if a not in self._cache:
self._cache[a] = self.reallyCalc(a)
return self._cache[a]
def reallyCalc(self, a):
return # a real answer
calcSomething = CalcSomething()
Now you can use calcSomething
as if it were a function. But it remains tidy and self-contained.
One option is to abuse default parameters. ie:
def CalcSomething(a, _cache={}):
if _cache.has_key(a):
This has the advantage that you don’t need to qualify the name, and will get fast local access to the variables rather than doing two slow dict lookups. However it still has the problem that it is mentioned outside the function (in fact it’s worse since its now in the function signature.)
To prevent this, a better solution would be to wrap the function in a closure containing your statics:
@apply
def CalcSomething():
cache = {} # statics go here
def CalcSomething(a):
if cache.has_key(a):
return cache[a]
cache[a] = ReallyCalc(a)
return cache[a]
return CalcSomething
The solution proposed by S.Lott is the solution I would propose too.
There are useful “memoize” decorators around, too, like:
- Memoize decorator function with cache size limit
- Memoize decorator with O(1) length-limited LRU cache, supports mutable types
Given all that, I’m providing an alternative for your initial attempt at a function and a “static local”, which is standalone:
def calc_something(a):
try:
return calc_something._cache[a]
except AttributeError: # _cache is not there
calc_something._cache= {}
except KeyError: # the result is not there
pass
# compute result here
calc_something._cache[a]= result
return result