Recursive type annotations
Question:
I’m trying to introduce static type annotations to my codebase where applicable. One case is when reading a JSON, the resulting object will be a dictionary keyed by strings, with values of one of the following types:
boolstrfloatintlistdict
However the list and dict above can contain that same sort of dictionary, leading to a recursive definition. Is this representable in Python3’s type structure?
Answers:
As of mypy 0.990, mypy finally supports recursive type annotations, using the natural syntax:
from typing import Union, Dict, List
JSONVal = Union[None, bool, str, float, int, List['JSONVal'], Dict[str, 'JSONVal']]
d: JSONVal = {'a': ['b']}
mypy output:
Success: no issues found in 1 source file
Before 0.990, this would produce an error reporting a lack of recursive type support:
$ mypy asdf.py
asdf.py:3: error: Recursive types not fully supported yet, nested types replaced with "Any"
On such versions, Dict[str, Any] would be the way to go.
You can also use mutually recursive type aliases now, so you can do things like
from typing import Union, Dict, List
JSONVal = Union[None, bool, str, float, int, 'JSONArray', 'JSONObject']
JSONArray = List[JSONVal]
JSONObject = Dict[str, JSONVal]
d: JSONObject = {'a': ['b']}
For more recent versions of mypy, this comment to the mentioned MyPy issue tracker suggests a partially working (but a bit convoluted) way of doing this using protocols, as long a using TypeVar is not required:
from __future__ import annotations
from collections.abc import Iterator
from typing import TypeVar, Protocol, overload, Any, TYPE_CHECKING
_T_co = TypeVar("_T_co")
class _RecursiveSequence(Protocol[_T_co]):
def __len__(self) -> int: ...
@overload
def __getitem__(self, __index: int) -> _T_co | _RecursiveSequence[_T_co]: ...
@overload
def __getitem__(self, __index: slice) -> _RecursiveSequence[_T_co]: ...
def __contains__(self, __x: object) -> bool: ...
def __iter__(self) -> Iterator[_T_co | _RecursiveSequence[_T_co]]: ...
def __reversed__(self) -> Iterator[_T_co | _RecursiveSequence[_T_co]]: ...
def count(self, __value: Any) -> int: ...
def index(self, __value: Any, __start: int = ..., __stop: int = ...) -> int: ...
def func1(a: _RecursiveSequence[int]) -> int: ...
if TYPE_CHECKING:
reveal_type(func1([1])) # Revealed type is "builtins.int"
reveal_type(func1([[1]])) # Revealed type is "builtins.int"
reveal_type(func1([[[1]]])) # Revealed type is "builtins.int"
reveal_type(func1((1, 2, 3))) # Revealed type is "builtins.int"
reveal_type(func1([(1, 2, 3)])) # Revealed type is "builtins.int"
reveal_type(func1([True])) # Revealed type is "builtins.int"
Support for recursive types is now in Mypy.
As of October 2022 the implementation is provisional. You can enable it by adding the enable_recursive_aliases = true flag to pyproject.toml.
Starting from version 0.990 this will be enabled by default. Source.
I’m trying to introduce static type annotations to my codebase where applicable. One case is when reading a JSON, the resulting object will be a dictionary keyed by strings, with values of one of the following types:
boolstrfloatintlistdict
However the list and dict above can contain that same sort of dictionary, leading to a recursive definition. Is this representable in Python3’s type structure?
As of mypy 0.990, mypy finally supports recursive type annotations, using the natural syntax:
from typing import Union, Dict, List
JSONVal = Union[None, bool, str, float, int, List['JSONVal'], Dict[str, 'JSONVal']]
d: JSONVal = {'a': ['b']}
mypy output:
Success: no issues found in 1 source file
Before 0.990, this would produce an error reporting a lack of recursive type support:
$ mypy asdf.py
asdf.py:3: error: Recursive types not fully supported yet, nested types replaced with "Any"
On such versions, Dict[str, Any] would be the way to go.
You can also use mutually recursive type aliases now, so you can do things like
from typing import Union, Dict, List
JSONVal = Union[None, bool, str, float, int, 'JSONArray', 'JSONObject']
JSONArray = List[JSONVal]
JSONObject = Dict[str, JSONVal]
d: JSONObject = {'a': ['b']}
For more recent versions of mypy, this comment to the mentioned MyPy issue tracker suggests a partially working (but a bit convoluted) way of doing this using protocols, as long a using TypeVar is not required:
from __future__ import annotations
from collections.abc import Iterator
from typing import TypeVar, Protocol, overload, Any, TYPE_CHECKING
_T_co = TypeVar("_T_co")
class _RecursiveSequence(Protocol[_T_co]):
def __len__(self) -> int: ...
@overload
def __getitem__(self, __index: int) -> _T_co | _RecursiveSequence[_T_co]: ...
@overload
def __getitem__(self, __index: slice) -> _RecursiveSequence[_T_co]: ...
def __contains__(self, __x: object) -> bool: ...
def __iter__(self) -> Iterator[_T_co | _RecursiveSequence[_T_co]]: ...
def __reversed__(self) -> Iterator[_T_co | _RecursiveSequence[_T_co]]: ...
def count(self, __value: Any) -> int: ...
def index(self, __value: Any, __start: int = ..., __stop: int = ...) -> int: ...
def func1(a: _RecursiveSequence[int]) -> int: ...
if TYPE_CHECKING:
reveal_type(func1([1])) # Revealed type is "builtins.int"
reveal_type(func1([[1]])) # Revealed type is "builtins.int"
reveal_type(func1([[[1]]])) # Revealed type is "builtins.int"
reveal_type(func1((1, 2, 3))) # Revealed type is "builtins.int"
reveal_type(func1([(1, 2, 3)])) # Revealed type is "builtins.int"
reveal_type(func1([True])) # Revealed type is "builtins.int"
Support for recursive types is now in Mypy.
As of October 2022 the implementation is provisional. You can enable it by adding the enable_recursive_aliases = true flag to pyproject.toml.
Starting from version 0.990 this will be enabled by default. Source.