Python typing: Use a class variable's value as return type of a (mixin) method

Question:

Summary

How can I use a class variable’s value (which is a class object) as the return type of a (mixin) method with Python typing / mypy?

Here is a minimal example, below you’ll find the real, more complicated use case:

from typing import Generic, Type, TypeVar


T = TypeVar('T')


class Base(Generic[T]):
    return_type: Type[T]
    value: T  # This attribute is only needed for this minimal example


class Mixin:
    def get(self: Base[T]) -> T:  # mypy: The erased type of self "Base" is not a supertype of its class "Mixin"
        return self.return_type(self.value)  # mypy: Too many arguments for "object"


class Concrete(Mixin, Base[int]):
    return_type = int

    def __init__(self):
        self.value = 3


c = Concrete()
x: str = c.get()  # mypy: expected incompatible types (str vs int) error :)

I can get rid of the second error when set Base as super class of Mixin, but that is not really what I want. I have now idea though, how I can properly define the return_type: Type[T].

I’ve read the python typing docs as well as the mypy docs, but found nothing. A web search also didn’t yield useful results.

What I really want to do

I am currently writing a REST client with an architecture similar to python-gitlab:

  • The users uses an ApiClient class that knows the API URL and does all HTTP requests.

  • The API’s endpoints are represented by REST manager classes that are attributes of the ApiClient. Depending on the endpoint’s functionality, a REST manager can list the endpoint’s objects, get a single object, or create, update and delete an object.

  • The RestManager returns and receives “dumb” data classes (e.g., attrs or pydantic models)

  • Concrete REST managers subclass a RestManager base class and various mixins for HTTP operations, e.g., a GetMixin for getting a single object by ID.

  • A concrete REST manager has a class variable that holds the class of the objects that it is going to return.

  • In the mixin classes, I want to express “this methods returns an instance of the object class, that the sub-classing restmanager defined as a class variable”.

Example usage:

client = ApiClient('https://example.com/myapi/v1')
item = client.items.get(42)
assert isinstance(item, Item)

Implementation:

from typing import ClassVar, Type, TypeVar


T = TypeVar(T)


class Item:
    """Data class that represents objects of the "items" endpoint"""
    pass


class ApiClient:
    """Main object that the user works with."""
    def __init__(self, url: str):
        self.url = url
        # There is one manager instance for each endpoint of the API
        self.items = ItemManager(self) 
        # self.cats = CatManager(self)

    def http_get(self, path: str) -> 'Response':
        ...  # Request the proper url and return a response object


class RestManager:
    """Base class for REST managers."""
    _path: ClassVar[str]
    _obj_cls: ClassVar[Type[T]]  # Concrete subclasses set this with an object class, e.g., "Item"

    def __init__(self, client: ApiClient):
        self.client = client

    @property
    def path(self) -> str:
        return self._path


class GetMixin:
    """Mixin for getting a single object by ID"""
    def get(self: RestManager, id: int) -> T:  # Return type is the value the subclass' "_obj_cls" attribute
        response = self.client.http_get(f'{self.path}/{id}')
        return self._obj_cls(**response.json())


class ItemsManager(GetMixin, RestManager):
    """Concrete manager for "Item" objects."""
    _path = '/items'
    _obj_cls = Item  # This is the return type of ItemsManager.get()


client = ApiClient()
item = client.items.get(42)
assert isinstance(item, Item)
Asked By: Stefan Scherfke

||

Answers:

Disclaimer: I haven’t read your real-world use case carefully so I could be wrong. The following analysis is based on your simplified example.

I don’t think this is supported by mypy. Currently mypy assumes (and rightfully so) that the type of self in a method is a subtype of the class.

However, for your mixin to work, there must be a restriction on the kind of classes that it can be mixed with. For example, in your simplified example, the class must have return_type and value attributes. My suggestion is that you can also add those as annotations for yout mixin class, resulting in something like:

T = TypeVar('T')


class Base(Generic[T]):
    return_type: Type[T]
    value: T  # This attribute is only needed for this minimal example


class Mixin(Generic[T]):
    return_type: Type[T]
    value: T

    def get(self) -> T:  # annotation on `self` can be dropped
        return self.return_type(self.value)

Note that the final line is still an error in mypy, because it cannot prove that the __init__ method of self.return_type takes only one argument. You can fix this with typing.Protocol which was introduced in Python 3.7, but it might be a bit of overkill. IMHO, if you have some simple code that you’re pretty sure is right, sometimes a little # type: ignore is best for you.

Answered By: Zecong Hu

I found a solution that works. It is not optimal, because the Mixin classes need to inherit from RestManager. But mypy can successfully deduce the expected return type.

The code requires Pyhton 3.10. With 3.11, you can import assert_type directly from typing. With older versions, you need to use typing.Type[T] instead of type[t].

from typing import ClassVar, Generic, TypeVar

from typing_extensions import assert_type


T = TypeVar("T")


class Item:
    """Data class that represents objects of the "items" endpoint"""


class ApiClient:
    """Main object that the user works with."""

    def __init__(self, url: str):
        self.url = url
        # There is one manager instance for each endpoint of the API
        self.items = ItemsManager(self)
        # self.cats = CatManager(self)

    def http_get(self, path: str) -> "Response":
        ...  # Request the proper url and return a response object


class RestManager(Generic[T]):
    """Base class for REST managers."""

    _path: ClassVar[str]
    _obj_cls: type[T]

    def __init__(self, client: ApiClient):
        self.client = client

    @property
    def path(self) -> str:
        return self._path


class GetMixin(RestManager, Generic[T]):
    """Mixin for getting a single object by ID"""

    def get(self, iid: int) -> T:
        response = self.client.http_get(f"{self.path}/{iid}")
        return self._obj_cls(**response.json())


class ItemsManager(GetMixin[Item], RestManager[Item]):
    """Concrete manager for "Item" objects."""

    _path = "/items"
    _obj_cls = Item


def main() -> None:
    client = ApiClient("api")
    item = client.items.get(42)
    assert_type(item, Item)
    assert isinstance(item, Item)


if __name__ == "__main__":
    main()
Answered By: Stefan Scherfke
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.