How can I use ParamSpec with method decorators?

Question

I was following the example from PEP 0612 (last one in the Motivation section) to create a decorator that can add default parameters to a function. The problem is, the example provided only works for functions but not methods, because Concate doesn’t allow inserting self anywhere in the definition.

Consider this example, as an adaptation of the one in the PEP:

def with_request(f: Callable[Concatenate[Request, P], R]) -> Callable[P, R]:
    def inner(*args: P.args, **kwargs: P.kwargs) -> R:
        return f(*args, request=Request(), **kwargs)

    return inner


class Thing:
    @with_request
    def takes_int_str(self, request: Request, x: int, y: str) -> int:
        print(request)
        return x + 7


thing = Thing()
thing.takes_int_str(1, "A")  # Invalid self argument "Thing" to attribute function "takes_int_str" with type "Callable[[str, int, str], int]"
thing.takes_int_str("B", 2)  # Argument 2 to "takes_int_str" of "Thing" has incompatible type "int"; expected "str"

Both attempts raise a mypy error because Request doesn’t match self as the first argument of the method, like Concatenate said. The problem is that Concatenate doesn’t allow you to append Request to the end, so something like Concatenate[P, Request] won’t work either.

This would be the ideal way to do this in my view, but it doesn’t work because "The last parameter to Concatenate needs to be a ParamSpec".

def with_request(f: Callable[Concatenate[P, Request], R]) -> Callable[P, R]:
    ...


class Thing:
    @with_request
    def takes_int_str(self, x: int, y: str, request: Request) -> int:
        ...

Any ideas?

Asked By: Ocab19

||

Source

Answer 1

There is surprisingly little about this online. I was able to find someone else’s discussion of this over at python/typing‘s Github, which I distilled using your example.

The crux of this solution is Callback Protocols, which are functionally equivalent to Callable, but additionally enable us to modify the return type of __get__ (essentially removing the self parameter) as is done for standard methods.

from __future__ import annotations

from typing import Any, Callable, Concatenate, Generic, ParamSpec, Protocol, TypeVar

from requests import Request

P = ParamSpec("P")
R = TypeVar("R", covariant=True)


class Method(Protocol, Generic[P, R]):
    def __get__(self, instance: Any, owner: type | None = None) -> Callable[P, R]:
        ...

    def __call__(self_, self: Any, *args: P.args, **kwargs: P.kwargs) -> R:
        ...


def request_wrapper(f: Callable[Concatenate[Any, Request, P], R]) -> Method[P, R]:
    def inner(self, *args: P.args, **kwargs: P.kwargs) -> R:
        return f(self, Request(), *args, **kwargs)

    return inner


class Thing:
    @request_wrapper
    def takes_int_str(self, request: Request, x: int, y: str) -> int:
        print(request)
        return x + 7


thing = Thing()
thing.takes_int_str(1, "a")

Since @Creris asked about the mypy error raised from the definition of inner, which is an apparent bug in mypy w/ ParamSpec and Callback Protocols as of mypy==0.991, here is an alternate implementation with no errors:

from __future__ import annotations

from typing import Any, Callable, Concatenate, ParamSpec, TypeVar

from requests import Request

P = ParamSpec("P")
R = TypeVar("R", covariant=True)


def request_wrapper(f: Callable[Concatenate[Any, Request, P], R]) -> Callable[Concatenate[Any, P], R]:
    def inner(self: Any, *args: P.args, **kwargs: P.kwargs) -> R:
        return f(self, Request(), *args, **kwargs)

    return inner


class Thing:
    @request_wrapper
    def takes_int_str(self, request: Request, x: int, y: str) -> int:
        print(request)
        return x + 7


thing = Thing()
thing.takes_int_str(1, "a")

Answered By: ringo

How can I use ParamSpec with method decorators?

Question:

Answers: