Python typing: Use a class variable's value as return type of a (mixin) method
Question:
Summary
How can I use a class variable’s value (which is a class object) as the return type of a (mixin) method with Python typing / mypy?
Here is a minimal example, below you’ll find the real, more complicated use case:
from typing import Generic, Type, TypeVar
T = TypeVar('T')
class Base(Generic[T]):
return_type: Type[T]
value: T # This attribute is only needed for this minimal example
class Mixin:
def get(self: Base[T]) -> T: # mypy: The erased type of self "Base" is not a supertype of its class "Mixin"
return self.return_type(self.value) # mypy: Too many arguments for "object"
class Concrete(Mixin, Base[int]):
return_type = int
def __init__(self):
self.value = 3
c = Concrete()
x: str = c.get() # mypy: expected incompatible types (str vs int) error :)
I can get rid of the second error when set Base
as super class of Mixin
, but that is not really what I want. I have now idea though, how I can properly define the return_type: Type[T]
.
I’ve read the python typing docs as well as the mypy docs, but found nothing. A web search also didn’t yield useful results.
What I really want to do
I am currently writing a REST client with an architecture similar to python-gitlab:
-
The users uses an ApiClient
class that knows the API URL and does all HTTP requests.
-
The API’s endpoints are represented by REST manager classes that are attributes of the ApiClient
. Depending on the endpoint’s functionality, a REST manager can list the endpoint’s objects, get a single object, or create, update and delete an object.
-
The RestManager returns and receives “dumb” data classes (e.g., attrs or pydantic models)
-
Concrete REST managers subclass a RestManager
base class and various mixins for HTTP operations, e.g., a GetMixin
for getting a single object by ID.
-
A concrete REST manager has a class variable that holds the class of the objects that it is going to return.
-
In the mixin classes, I want to express “this methods returns an instance of the object class, that the sub-classing restmanager defined as a class variable”.
Example usage:
client = ApiClient('https://example.com/myapi/v1')
item = client.items.get(42)
assert isinstance(item, Item)
Implementation:
from typing import ClassVar, Type, TypeVar
T = TypeVar(T)
class Item:
"""Data class that represents objects of the "items" endpoint"""
pass
class ApiClient:
"""Main object that the user works with."""
def __init__(self, url: str):
self.url = url
# There is one manager instance for each endpoint of the API
self.items = ItemManager(self)
# self.cats = CatManager(self)
def http_get(self, path: str) -> 'Response':
... # Request the proper url and return a response object
class RestManager:
"""Base class for REST managers."""
_path: ClassVar[str]
_obj_cls: ClassVar[Type[T]] # Concrete subclasses set this with an object class, e.g., "Item"
def __init__(self, client: ApiClient):
self.client = client
@property
def path(self) -> str:
return self._path
class GetMixin:
"""Mixin for getting a single object by ID"""
def get(self: RestManager, id: int) -> T: # Return type is the value the subclass' "_obj_cls" attribute
response = self.client.http_get(f'{self.path}/{id}')
return self._obj_cls(**response.json())
class ItemsManager(GetMixin, RestManager):
"""Concrete manager for "Item" objects."""
_path = '/items'
_obj_cls = Item # This is the return type of ItemsManager.get()
client = ApiClient()
item = client.items.get(42)
assert isinstance(item, Item)
Answers:
Disclaimer: I haven’t read your real-world use case carefully so I could be wrong. The following analysis is based on your simplified example.
I don’t think this is supported by mypy
. Currently mypy
assumes (and rightfully so) that the type of self
in a method is a subtype of the class.
However, for your mixin to work, there must be a restriction on the kind of classes that it can be mixed with. For example, in your simplified example, the class must have return_type
and value
attributes. My suggestion is that you can also add those as annotations for yout mixin class, resulting in something like:
T = TypeVar('T')
class Base(Generic[T]):
return_type: Type[T]
value: T # This attribute is only needed for this minimal example
class Mixin(Generic[T]):
return_type: Type[T]
value: T
def get(self) -> T: # annotation on `self` can be dropped
return self.return_type(self.value)
Note that the final line is still an error in mypy
, because it cannot prove that the __init__
method of self.return_type
takes only one argument. You can fix this with typing.Protocol
which was introduced in Python 3.7, but it might be a bit of overkill. IMHO, if you have some simple code that you’re pretty sure is right, sometimes a little # type: ignore
is best for you.
I found a solution that works. It is not optimal, because the Mixin classes need to inherit from RestManager
. But mypy can successfully deduce the expected return type.
The code requires Pyhton 3.10. With 3.11, you can import assert_type
directly from typing
. With older versions, you need to use typing.Type[T]
instead of type[t]
.
from typing import ClassVar, Generic, TypeVar
from typing_extensions import assert_type
T = TypeVar("T")
class Item:
"""Data class that represents objects of the "items" endpoint"""
class ApiClient:
"""Main object that the user works with."""
def __init__(self, url: str):
self.url = url
# There is one manager instance for each endpoint of the API
self.items = ItemsManager(self)
# self.cats = CatManager(self)
def http_get(self, path: str) -> "Response":
... # Request the proper url and return a response object
class RestManager(Generic[T]):
"""Base class for REST managers."""
_path: ClassVar[str]
_obj_cls: type[T]
def __init__(self, client: ApiClient):
self.client = client
@property
def path(self) -> str:
return self._path
class GetMixin(RestManager, Generic[T]):
"""Mixin for getting a single object by ID"""
def get(self, iid: int) -> T:
response = self.client.http_get(f"{self.path}/{iid}")
return self._obj_cls(**response.json())
class ItemsManager(GetMixin[Item], RestManager[Item]):
"""Concrete manager for "Item" objects."""
_path = "/items"
_obj_cls = Item
def main() -> None:
client = ApiClient("api")
item = client.items.get(42)
assert_type(item, Item)
assert isinstance(item, Item)
if __name__ == "__main__":
main()
Summary
How can I use a class variable’s value (which is a class object) as the return type of a (mixin) method with Python typing / mypy?
Here is a minimal example, below you’ll find the real, more complicated use case:
from typing import Generic, Type, TypeVar
T = TypeVar('T')
class Base(Generic[T]):
return_type: Type[T]
value: T # This attribute is only needed for this minimal example
class Mixin:
def get(self: Base[T]) -> T: # mypy: The erased type of self "Base" is not a supertype of its class "Mixin"
return self.return_type(self.value) # mypy: Too many arguments for "object"
class Concrete(Mixin, Base[int]):
return_type = int
def __init__(self):
self.value = 3
c = Concrete()
x: str = c.get() # mypy: expected incompatible types (str vs int) error :)
I can get rid of the second error when set Base
as super class of Mixin
, but that is not really what I want. I have now idea though, how I can properly define the return_type: Type[T]
.
I’ve read the python typing docs as well as the mypy docs, but found nothing. A web search also didn’t yield useful results.
What I really want to do
I am currently writing a REST client with an architecture similar to python-gitlab:
-
The users uses an
ApiClient
class that knows the API URL and does all HTTP requests. -
The API’s endpoints are represented by REST manager classes that are attributes of the
ApiClient
. Depending on the endpoint’s functionality, a REST manager can list the endpoint’s objects, get a single object, or create, update and delete an object. -
The RestManager returns and receives “dumb” data classes (e.g., attrs or pydantic models)
-
Concrete REST managers subclass a
RestManager
base class and various mixins for HTTP operations, e.g., aGetMixin
for getting a single object by ID. -
A concrete REST manager has a class variable that holds the class of the objects that it is going to return.
-
In the mixin classes, I want to express “this methods returns an instance of the object class, that the sub-classing restmanager defined as a class variable”.
Example usage:
client = ApiClient('https://example.com/myapi/v1')
item = client.items.get(42)
assert isinstance(item, Item)
Implementation:
from typing import ClassVar, Type, TypeVar
T = TypeVar(T)
class Item:
"""Data class that represents objects of the "items" endpoint"""
pass
class ApiClient:
"""Main object that the user works with."""
def __init__(self, url: str):
self.url = url
# There is one manager instance for each endpoint of the API
self.items = ItemManager(self)
# self.cats = CatManager(self)
def http_get(self, path: str) -> 'Response':
... # Request the proper url and return a response object
class RestManager:
"""Base class for REST managers."""
_path: ClassVar[str]
_obj_cls: ClassVar[Type[T]] # Concrete subclasses set this with an object class, e.g., "Item"
def __init__(self, client: ApiClient):
self.client = client
@property
def path(self) -> str:
return self._path
class GetMixin:
"""Mixin for getting a single object by ID"""
def get(self: RestManager, id: int) -> T: # Return type is the value the subclass' "_obj_cls" attribute
response = self.client.http_get(f'{self.path}/{id}')
return self._obj_cls(**response.json())
class ItemsManager(GetMixin, RestManager):
"""Concrete manager for "Item" objects."""
_path = '/items'
_obj_cls = Item # This is the return type of ItemsManager.get()
client = ApiClient()
item = client.items.get(42)
assert isinstance(item, Item)
Disclaimer: I haven’t read your real-world use case carefully so I could be wrong. The following analysis is based on your simplified example.
I don’t think this is supported by mypy
. Currently mypy
assumes (and rightfully so) that the type of self
in a method is a subtype of the class.
However, for your mixin to work, there must be a restriction on the kind of classes that it can be mixed with. For example, in your simplified example, the class must have return_type
and value
attributes. My suggestion is that you can also add those as annotations for yout mixin class, resulting in something like:
T = TypeVar('T')
class Base(Generic[T]):
return_type: Type[T]
value: T # This attribute is only needed for this minimal example
class Mixin(Generic[T]):
return_type: Type[T]
value: T
def get(self) -> T: # annotation on `self` can be dropped
return self.return_type(self.value)
Note that the final line is still an error in mypy
, because it cannot prove that the __init__
method of self.return_type
takes only one argument. You can fix this with typing.Protocol
which was introduced in Python 3.7, but it might be a bit of overkill. IMHO, if you have some simple code that you’re pretty sure is right, sometimes a little # type: ignore
is best for you.
I found a solution that works. It is not optimal, because the Mixin classes need to inherit from RestManager
. But mypy can successfully deduce the expected return type.
The code requires Pyhton 3.10. With 3.11, you can import assert_type
directly from typing
. With older versions, you need to use typing.Type[T]
instead of type[t]
.
from typing import ClassVar, Generic, TypeVar
from typing_extensions import assert_type
T = TypeVar("T")
class Item:
"""Data class that represents objects of the "items" endpoint"""
class ApiClient:
"""Main object that the user works with."""
def __init__(self, url: str):
self.url = url
# There is one manager instance for each endpoint of the API
self.items = ItemsManager(self)
# self.cats = CatManager(self)
def http_get(self, path: str) -> "Response":
... # Request the proper url and return a response object
class RestManager(Generic[T]):
"""Base class for REST managers."""
_path: ClassVar[str]
_obj_cls: type[T]
def __init__(self, client: ApiClient):
self.client = client
@property
def path(self) -> str:
return self._path
class GetMixin(RestManager, Generic[T]):
"""Mixin for getting a single object by ID"""
def get(self, iid: int) -> T:
response = self.client.http_get(f"{self.path}/{iid}")
return self._obj_cls(**response.json())
class ItemsManager(GetMixin[Item], RestManager[Item]):
"""Concrete manager for "Item" objects."""
_path = "/items"
_obj_cls = Item
def main() -> None:
client = ApiClient("api")
item = client.items.get(42)
assert_type(item, Item)
assert isinstance(item, Item)
if __name__ == "__main__":
main()