Why does Python's functools.partial definition use function attributes?

Question

In the functools.partial example definition, function attributes are used in the returned partial function object:

def partial(func, /, *args, **keywords):
    def newfunc(*fargs, **fkeywords):
        newkeywords = {**keywords, **fkeywords}
        return func(*args, *fargs, **newkeywords)
    newfunc.func = func
    newfunc.args = args
    newfunc.keywords = keywords
    return newfunc

What would be the potential problems if those attributes were removed, like below? Shouldn’t Python still keep the references to func, args and keywords as long as newfunc is alive?

def partial(func, /, *args, **keywords):
    def newfunc(*fargs, **fkeywords):
        newkeywords = {**keywords, **fkeywords}
        return func(*args, *fargs, **newkeywords)
    return newfunc

I know that the actual implementation of functools.partial may be completely different, but what if it were implemented this way, why are the attributes necessary?

Asked By: mindoverflow

||

Source

Answer 1

Since partial is actually a type, it would make more sense to provide an equivalent class, rather than turning it into a function.

class partial:
    def __init__(self, func, /, *args, **kwargs):
        self.func = func
        self.args = args
        self.keywords = kwargs

    def __call__(self, *args, **kwargs):
        new_keywords = {**self.keywords, **kwargs}
        return self.func(*self.args, *args, **new_keywords)

Note that everything that was assigned to the function attributes are now assigned to instance attributes of self. When you call an instance of partial, you call the original function, with the saved arguments merged with the "immediate" arguments.

def foo(a, b, c):
    ...


p = partial(foo, 3, c=9)
p(5)  # Equivalent to foo(3, 5, 9)

As to why the documentation uses a function, perhaps it was thought that a nested function would be simpler than explaining what __call__ does?

The function uses function attributes instead of using a closure precisely because the actual implementation of partial stores the saved arguments in attributes named args and keywords as well. They are trying to simulate the real type, after all.

Answered By: chepner

Answer 2

Apparently the function attributes aren’t actually necessary in the implementation of a partial function. They are only added because they are required by the API. There are two sets of variables here: one is in the newfunc’s closure, and another is in the form of attributes, which are not used by newfunc.

Thanks to @juanpa.arrivillaga for clarification.

Answered By: mindoverflow

Answer 3

Any implementation needs the function and its canned arguments stored somewhere. In your two examples you use a closure. When these partial implementations return, those values are added to the closure that is returned. In that case, .func, .args and .keywords are redundant. The closure already has them. So, go with option 2.

But python creates a class for this work instead. .func, .args and .keywords are the only copies of the needed data. The class implementation lets you make a partial of a partial in an efficient manner (it combines the arguments of the two partials) and is a bit tidier when building a string display of the object.

But your second closure mechanism is fine, just a different way of doing things.

Answered By: tdelaney

Why does Python's functools.partial definition use function attributes?

Question:

Answers: