Python __call__ special method practical example

This example uses memoization, basically storing values in a table (dictionary in this case) so you can look them up later instead of recalculating them.

Here we use a simple class with a __call__ method to calculate factorials (through a callable object) instead of a factorial function that contains a static variable (as that’s not possible in Python).

class Factorial:
    def __init__(self):
        self.cache = {}
    def __call__(self, n):
        if n not in self.cache:
            if n == 0:
                self.cache[n] = 1
            else:
                self.cache[n] = n * self.__call__(n-1)
        return self.cache[n]

fact = Factorial()

Now you have a fact object which is callable, just like every other function. For example

for i in xrange(10):                                                             
    print("{}! = {}".format(i, fact(i)))

# output
0! = 1
1! = 1
2! = 2
3! = 6
4! = 24
5! = 120
6! = 720
7! = 5040
8! = 40320
9! = 362880

And it is also stateful.

I find it useful because it allows me to create APIs that are easy to use (you have some callable object that requires some specific arguments), and are easy to implement because you can use Object Oriented practices.

The following is code I wrote yesterday that makes a version of the hashlib.foo methods that hash entire files rather than strings:

# filehash.py
import hashlib


class Hasher(object):
    """
    A wrapper around the hashlib hash algorithms that allows an entire file to
    be hashed in a chunked manner.
    """
    def __init__(self, algorithm):
        self.algorithm = algorithm

    def __call__(self, file):
        hash = self.algorithm()
        with open(file, 'rb') as f:
            for chunk in iter(lambda: f.read(4096), ''):
                hash.update(chunk)
        return hash.hexdigest()


md5    = Hasher(hashlib.md5)
sha1   = Hasher(hashlib.sha1)
sha224 = Hasher(hashlib.sha224)
sha256 = Hasher(hashlib.sha256)
sha384 = Hasher(hashlib.sha384)
sha512 = Hasher(hashlib.sha512)

This implementation allows me to use the functions in a similar fashion to the hashlib.foo functions:

from filehash import sha1
print sha1('somefile.txt')

Of course I could have implemented it a different way, but in this case it seemed like a simple approach.

Yes, when you know you’re dealing with objects, it’s perfectly possible (and in many cases advisable) to use an explicit method call. However, sometimes you deal with code that expects callable objects – typically functions, but thanks to __call__ you can build more complex objects, with instance data and more methods to delegate repetitive tasks, etc. that are still callable.

Also, sometimes you’re using both objects for complex tasks (where it makes sense to write a dedicated class) and objects for simple tasks (that already exist in functions, or are more easily written as functions). To have a common interface, you either have to write tiny classes wrapping those functions with the expected interface, or you keep the functions functions and make the more complex objects callable. Let’s take threads as example. The Thread objects from the standard libary module threading want a callable as target argument (i.e. as action to be done in the new thread). With a callable object, you are not restricted to functions, you can pass other objects as well, such as a relatively complex worker that gets tasks to do from other threads and executes them sequentially:

class Worker(object):
    def __init__(self, *args, **kwargs):
        self.queue = queue.Queue()
        self.args = args
        self.kwargs = kwargs

    def add_task(self, task):
        self.queue.put(task)

    def __call__(self):
        while True:
            next_action = self.queue.get()
            success = next_action(*self.args, **self.kwargs)
            if not success:
               self.add_task(next_action)

This is just an example off the top of my head, but I think it is already complex enough to warrant the class. Doing this only with functions is hard, at least it requires returning two functions and that’s slowly getting complex. One could rename __call__ to something else and pass a bound method, but that makes the code creating the thread slightly less obvious, and doesn’t add any value.

Class-based decorators use __call__ to reference the wrapped function. E.g.:

class Deco(object):
    def __init__(self,f):
        self.f = f
    def __call__(self, *args, **kwargs):
        print args
        print kwargs
        self.f(*args, **kwargs)

There is a good description of the various options here at Artima.com

IMHO __call__ method and closures give us a natural way to create STRATEGY design pattern in Python. We define a family of algorithms, encapsulate each one, make them interchangeable and in the end we can execute a common set of steps and, for example, calculate a hash for a file.

Specify a __metaclass__ and override the __call__ method, and have the specified meta classes’ __new__ method return an instance of the class, viola you have a “function” with methods.

__call__ is also used to implement decorator classes in python. In this case the instance of the class is called when the method with the decorator is called.

class EnterExitParam(object):

    def __init__(self, p1):
        self.p1 = p1

    def __call__(self, f):
        def new_f():
            print("Entering", f.__name__)
            print("p1=", self.p1)
            f()
            print("Leaving", f.__name__)
        return new_f


@EnterExitParam("foo bar")
def hello():
    print("Hello")


if __name__ == "__main__":
    hello()

program output:

Entering hello
p1= foo bar
Hello
Leaving hello

I just stumbled upon a usage of __call__() in concert with __getattr__() which I think is beautiful. It allows you to hide multiple levels of a JSON/HTTP/(however_serialized) API inside an object.

The __getattr__() part takes care of iteratively returning a modified instance of the same class, filling in one more attribute at a time. Then, after all information has been exhausted, __call__() takes over with whatever arguments you passed in.

Using this model, you can for example make a call like api.v2.volumes.ssd.update(size=20), which ends up in a PUT request to https://some.tld/api/v2/volumes/ssd/update.

The particular code is a block storage driver for a certain volume backend in OpenStack, you can check it out here: https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/nexenta/jsonrpc.py

EDIT: Updated the link to point to master revision.

We can use __call__ method to use other class methods as static methods.

    class _Callable:
        def __init__(self, anycallable):
            self.__call__ = anycallable

    class Model:

        def get_instance(conn, table_name):

            """ do something"""

        get_instance = _Callable(get_instance)

    provs_fac = Model.get_instance(connection, "users")             

One common example is the __call__ in functools.partial, here is a simplified version (with Python >= 3.5):

class partial:
    """New function with partial application of the given arguments and keywords."""

    def __new__(cls, func, *args, **kwargs):
        if not callable(func):
            raise TypeError("the first argument must be callable")
        self = super().__new__(cls)

        self.func = func
        self.args = args
        self.kwargs = kwargs
        return self

    def __call__(self, *args, **kwargs):
        return self.func(*self.args, *args, **self.kwargs, **kwargs)

Usage:

def add(x, y):
    return x + y

inc = partial(add, y=1)
print(inc(41))  # 42

The function call operator.

class Foo:
    def __call__(self, a, b, c):
        # do something

x = Foo()
x(1, 2, 3)

The __call__ method can be used to redefined/re-initialize the same object. It also facilitates the use of instances/objects of a class as functions by passing arguments to the objects.

I find a good place to use callable objects, those that define __call__(), is when using the functional programming capabilities in Python, such as map(), filter(), reduce().

The best time to use a callable object over a plain function or a lambda function is when the logic is complex and needs to retain some state or uses other info that in not passed to the __call__() function.

Here’s some code that filters file names based upon their filename extension using a callable object and filter().

Callable:

import os

class FileAcceptor(object):
    def __init__(self, accepted_extensions):
        self.accepted_extensions = accepted_extensions

    def __call__(self, filename):
        base, ext = os.path.splitext(filename)
        return ext in self.accepted_extensions

class ImageFileAcceptor(FileAcceptor):
    def __init__(self):
        image_extensions = ('.jpg', '.jpeg', '.gif', '.bmp')
        super(ImageFileAcceptor, self).__init__(image_extensions)

Usage:

filenames = [
    'me.jpg',
    'me.txt',
    'friend1.jpg',
    'friend2.bmp',
    'you.jpeg',
    'you.xml']

acceptor = ImageFileAcceptor()
image_filenames = filter(acceptor, filenames)
print image_filenames

Output:

['me.jpg', 'friend1.jpg', 'friend2.bmp', 'you.jpeg']

This is too late but I’m giving an example. Imagine you have a Vector class and a Point class. Both take x, y as positional args. Let’s imagine you want to create a function that moves the point to be put on the vector.

4 Solutions

• put_point_on_vec(point, vec)

• Make it a method on the vector class. e.g my_vec.put_point(point)

• Make it a method on the Point class. my_point.put_on_vec(vec)
• Vector implements __call__, So you can use it like my_vec_instance(point)

This is actually part of some examples I’m working on for a guide for dunder methods explained with Maths that I’m gonna release sooner or later.

I left the logic of moving the point itself because this is not what this question is about

import random


class Bingo:

def __init__(self,items):
    self._items=list(items)
    random.shuffle(self._items,random=None)


def pick(self):
    try:
        return self._items.pop()
    except IndexError:
        raise LookupError('It is empty now!')

def __call__(self):
    return self.pick()

b= Bingo(range(3))
print(b.pick())
print(b())
print(callable(b))

Now the output can be..(As first two answer keeps shuffling between [0,3])

2

1

True

You can see that Class Bingo implements _call_ method, an easy way to create a function like objects that have an internal state that must be kept across invocations like remaining items in Bingo. Another good use case of _call_ is Decorators. Decorators must be callable and it is sometimes convenient to "remember"
something between calls of decorators. (i.e., memoization- caching the results of expensive computation for later use.)

I’m a novice, but here is my take: having __call__ makes composition easier to code. If f, g are instance of a class Function that has a method eval(self,x), then with __call___ one could write f(g(x)) as opposed to f.eval(g.eval(x)).

A neural network can be composed from smaller neural networks, and in pytorch we have a __call__ in the Module class:

Here’s an example of where __call__ is used in practice: in pytorch, when defining a neural network (call it class MyNN(nn.Module) for example) as a subclass of torch.nn.module.Module, one typically defines a forward method for the class, but typically when applying an input tensor x to an instance model=MyNN() we just write model(x) as opposed to model.forward(x) but both give the same answer. If you dig into the source for torch.nn.module.Module here

https://pytorch.org/docs/stable/_modules/torch/nn/modules/module.html#Module

and search for __call__ one can eventually trace it back – at least in some cases – to a call to self.forward

import torch.nn as nn
import torch.nn.functional as F

class MyNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.layer1 = nn.Linear(784, 200)



    def forward(self, x):
        return F.ReLU(self.layer1(x))

x=torch.rand(10,784)
model=MyNN()
print(model(x))
print(model.forward(x))

will print the same values in the last two lines as the Module class has implemented __call___ so that’s what I believe Python turns to when it sees model(x), and the __call__ which in turn eventually calls model.forward(x))