Type annotations for abstract classes that are coupled by a shared, arbitrary type

Question

(I’m rather new to Python’s type annotations and mypy, so I’m describing my problem in detail in order to avoid running into an XY problem)

I have two abstract classes that exchange values of an arbitrary but fixed type:

from __future__ import annotations

from abc import ABC, abstractmethod
from typing import Generic, TypeVar


T = TypeVar('T')  # result type


class Command(ABC, Generic[T]):
    @abstractmethod
    def execute(self, runner: Runner[T]) -> T:
        raise NotImplementedError()


class Runner(ABC, Generic[T]):
    def run(self, command: Command[T]) -> T:
        return command.execute(self)

In my implementation of this interface, the Command subclass needs to access an attribute of my Runner subclass (imagine that the command can adapt to runners with different capabilities):

class MyCommand(Command[bool]):
    def execute(self, runner: Runner[bool]) -> bool:
        # Pseudo code to illustrate dependency on runner's attributes
        return runner.magic_level > 10


class MyRunner(Runner[bool]):
    magic_level: int = 20

This works as expected, but doesn’t satisfy mypy:

mypy_sandbox.py:24: error: "Runner[bool]" has no attribute "magic_level"  [attr-defined]

Obviously, mypy is correct: the magic_level attribute is defined in MyRunner, but not in Runner (which is the type of the argument to execute). So the interface is too generic — a command doesn’t need to work with any runner, only with some runners. So let’s make Command generic on a second type var, to capture the supported runner class:

R = TypeVar('R')  # runner type
T = TypeVar('T')  # result type


class Command(ABC, Generic[T, R]):
    @abstractmethod
    def execute(self, runner: R) -> T:
        raise NotImplementedError()


class Runner(ABC, Generic[T]):
    def run(self, command: Command[T, Runner[T]]) -> T:
        return command.execute(self)


class MyCommand(Command[bool, MyRunner]):
    def execute(self, runner: MyRunner) -> bool:
        # Pseudo code to illustrate dependency on runner's attributes
        return runner.magic_level > 10


# MyRunner defined as before

This satisfies mypy, but when I try to use the code, mypy complains again:

if __name__ == '__main__':
    command = MyCommand()
    runner = MyRunner()
    print(runner.run(command))

mypy_sandbox.py:35: error: Argument 1 to "run" of "Runner" has incompatible type "MyCommand"; expected "Command[bool, Runner[bool]]"  [arg-type]

This time I don’t even understand the error: MyCommand is a subclass of Command[bool, MyRunner], and MyRunner is a subclass of Runner[bool], so why is MyCommand incompatible with Command[bool, Runner[bool]]?

And if mypy was satisfied I could probably implement a Command subclass with a Runner subclass that uses "a different value" for T (since R is not tied to T) without mypy complaining. I tried R = TypeVar('R', bound='Runner[T]'), but that throws yet another error:

error: Type variable "mypy_sandbox.T" is unbound  [valid-type]

How can I type-annotate this so that extensions as described above are possible but still type-checked correctly?

Asked By: Florian Brucker

||

Source

Answer 1

The current annotations are indeed a contradiction:

The Runner allows only Commands of the form Command[T, Runner[T]].
The execute method of Command[bool, Runner[bool]] accepts any Runner[bool].
The execute method of MyCommand only accepts any "Runner[bool] with a magic_level".

Therefore, MyCommand is not a Command[bool, Runner[bool]] – it does not accept any "Runner[bool] without a magic_level". This forces MyPy to reject the substitution, even though the reason for it happens earlier.

This issue can be solved by parameterising over R as the self-type of Runner. This avoids forcing Runner to parameterise Command by the baseclass Runner[T], and instead parameterises it by the actual subtype of Runner[T].

R = TypeVar('R', bound='Runner[Any]')
T = TypeVar('T')  # result type

class Command(ABC, Generic[T, R]):
    @abstractmethod
    def execute(self, runner: R) -> T:
        raise NotImplementedError()


# Runner is not generic in R
class Runner(ABC, Generic[T]):
    # Runner.run is generic in its owner
    def run(self: R, command: Command[T, R]) -> T:
        return command.execute(self)

Answered By: MisterMiyagi

Answer 2

You need to add the instance variable to the Runner interface:

from __future__ import annotations

from abc import ABC, abstractmethod
from typing import Generic, TypeVar

T = TypeVar('T')


class Command(ABC, Generic[T]):
    @abstractmethod
    def execute(self, runner: Runner[T]) -> T:
        pass


class Runner(ABC, Generic[T]):
    magic_level: int   # <- note the change to your code here!

    def run(self, command: Command[T]) -> T:
        return command.execute(self)

then your first implementation will run without errors. I feel that you also had an interface in mind that included the instance variable. Of course, the whole thing also works if magic_level is a TypeVar too or if it is characterized by a more open Protocol.

Answered By: Ingo

Type annotations for abstract classes that are coupled by a shared, arbitrary type

Question:

Answers: