How to implement a lazy setdefault?

Question

One minor annoyance with dict.setdefault is that it always evaluates its second argument (when given, of course), even when the first the first argument is already a key in the dictionary.

For example:

import random
def noisy_default():
    ret = random.randint(0, 10000000)
    print 'noisy_default: returning %d' % ret
    return ret

d = dict()
print d.setdefault(1, noisy_default())
print d.setdefault(1, noisy_default())

This produces ouptut like the following:

noisy_default: returning 4063267
4063267
noisy_default: returning 628989
4063267

As the last line confirms, the second execution of noisy_default is unnecessary, since by this point the key 1 is already present in d (with value 4063267).

Is it possible to implement a subclass of dict whose setdefault method evaluates its second argument lazily?

EDIT:

Below is an implementation inspired by BrenBarn’s comment and Pavel Anossov’s answer. While at it, I went ahead and implemented a lazy version of get as well, since the underlying idea is essentially the same.

class LazyDict(dict):
    def get(self, key, thunk=None):
        return (self[key] if key in self else
                thunk() if callable(thunk) else
                thunk)


    def setdefault(self, key, thunk=None):
        return (self[key] if key in self else
                dict.setdefault(self, key,
                                thunk() if callable(thunk) else
                                thunk))

Now, the snippet

d = LazyDict()
print d.setdefault(1, noisy_default)
print d.setdefault(1, noisy_default)

produces output like this:

noisy_default: returning 5025427
5025427
5025427

Notice that the second argument to d.setdefault above is now a callable, not a function call.

When the second argument to LazyDict.get or LazyDict.setdefault is not a callable, they behave the same way as the corresponding dict methods.

If one wants to pass a callable as the default value itself (i.e., not meant to be called), or if the callable to be called requires arguments, prepend lambda: to the appropriate argument. E.g.:

d1.setdefault('div', lambda: div_callback)

d2.setdefault('foo', lambda: bar('frobozz'))

Those who don’t like the idea of overriding get and setdefault, and/or the resulting need to test for callability, etc., can use this version instead:

class LazyButHonestDict(dict):
    def lazyget(self, key, thunk=lambda: None):
        return self[key] if key in self else thunk()


    def lazysetdefault(self, key, thunk=lambda: None):
        return (self[key] if key in self else
                self.setdefault(key, thunk()))

Asked By: kjo

||

Source

Answer 1

No, evaluation of arguments happens before the call. You can implement a setdefault-like function that takes a callable as its second argument and calls it only if it is needed.

Answered By: Pavel Anossov

Answer 2

This can be accomplished with defaultdict, too. It is instantiated with a callable which is then called when a nonexisting element is accessed.

from collections import defaultdict

d = defaultdict(noisy_default)
d[1] # noise
d[1] # no noise

The caveat with defaultdict is that the callable gets no arguments, so you can not derive the default value from the key as you could with dict.setdefault. This can be mitigated by overriding __missing__ in a subclass:

from collections import defaultdict

class defaultdict2(defaultdict):
    def __missing__(self, key):
        value = self.default_factory(key)
        self[key] = value
        return value

def noisy_default_with_key(key):
    print key
    return key + 1

d = defaultdict2(noisy_default_with_key)
d[1] # prints 1, sets 2, returns 2
d[1] # does not print anything, does not set anything, returns 2

For more information, see the collections module.

Answered By: Santtu Pajukanta

Answer 3

You can do that in a one-liner using a ternary operator:

value = cache[key] if key in cache else cache.setdefault(key, func(key))

If you are sure that the cache will never store falsy values, you can simplify it a little bit:

value = cache.get(key) or cache.setdefault(key, func(key))

Answered By: Cesar Canassa

Answer 4

There seems to be no one-liner that doesn’t require an extra class or extra lookups. For the record, here is a easy (even not concise) way of achieving that without either of them.

try:
    value = dct[key]
except KeyError:
    value = noisy_default()
    dct[key] = value
return value

Answered By: Dennis Golomazov

How to implement a lazy setdefault?

Question:

Answers: