Why doesn't parameter type "Dict[str, Union[str, int]]" accept value of type "Dict[str, str]" (mypy)
Question:
I have a type for a dictionary of variables passed to a template:
VariablesDict = Dict[str, Union[int, float, str, None]]
Basically, any dictionary where the keys are strings and the values are strings, numbers or None. I use this type in several template related functions.
Take this example function:
def render_template(name: str, variables: VariablesDict):
...
Calling this function with a dictionary literal works fine:
render_template("foo", {"key": "value"})
However, if I assign the dictionary to a variable first, like this:
variables = {"key": "value"}
render_template("foo", variables)
Mypy gives an error:
Argument 2 to "render_template" has incompatible type "Dict[str, str]"; expected "Dict[str, Union[int, float, str, None]]"
It seems to me that any value of type Dict[str, str] should be safe to pass to a function that expects a parameter of type Dict[str, Union[int, float, str, None]]. Why doesn’t that work by default? Is there anything I can do to make this work?
Answers:
One minor edit gets this to work:
variables: VariablesDict = {"key": "value"}
The MyPy docs list three solutions to the invariance problem:
- Use an explicit type annotation (that is my answer shown above).
- Make a copy of the right hand side (this is egregious).
- Use immutable collections as annotations whenever possible: (that matches the excellent answer from @Samwise).
FWIW, the MyPy error message in this case is pretty good. It first gives the difference between the inferred type and the expected type, then it refers you to the relevant section of the docs, and lastly, it makes an appropriate suggestion:
tmp12.py:12: error: Argument 2 to "render_template" has incompatible type "Dict[str, str]"; expected "Dict[str, Union[int, float, st
r, None]]"
tmp12.py:12: note: "Dict" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
tmp12.py:12: note: Consider using "Mapping" instead, which is covariant in the value type
The reason it doesn’t work is that Dict is mutable, and a function which accepts a Dict[str, int|float|str|None] could therefore reasonably insert any of those types into its argument. If the argument was actually a Dict[str, str], it now contains values that violate its type. (For more on this, google "covariance/contravariance/invariance" and "Liskov Substitution Principle" — as a general rule, mutable containers are invariant over their generic type[s].)
As long as render_template doesn’t need to modify the dict you pass to it, an easy fix is to have it take a Mapping (which is an abstract supertype of dict that doesn’t imply mutability, and is therefore covariant) instead of a Dict:
def render_template(name: str, variables: Mapping[str, Union[int, float, str, None]]):
...
It seems to me that any value of type Dict[str, str] should be safe to pass to a function that expects a parameter of type Dict[str, Union[int, float, str, None]].
Unfortunately, that assumption is wrong. Because dict is mutable, it must be invariant in its second type parameter.
Why doesn’t that work by default?
It doesn’t work because it is not type-safe: render_template expects an argument of type dict[str, Union[int, float, str, None]], which means that (among other things), it expects to be able to store an int in the dict.
However, you cannot store an int in a dict[str, str], therefore MyPy rightfully rejects the code.
Is there anything I can do to make this work?
The difference between the two snippets of code is that type inference in MyPy is local. Therefore, in the second example, MyPy does not take into account how the variable variables is used later on. It only infers its type from the assignment expression, and it tries to infer the most strict type possible.
Now, as I mentioned above, the problem is that dict is mutable. Therefore, if you can make it immutable, your code will work.
Thankfully, there is just the type for you: Mapping. Mappings are immutable, therefore, they can be covariant in their second type parameter.
I have a type for a dictionary of variables passed to a template:
VariablesDict = Dict[str, Union[int, float, str, None]]
Basically, any dictionary where the keys are strings and the values are strings, numbers or None. I use this type in several template related functions.
Take this example function:
def render_template(name: str, variables: VariablesDict):
...
Calling this function with a dictionary literal works fine:
render_template("foo", {"key": "value"})
However, if I assign the dictionary to a variable first, like this:
variables = {"key": "value"}
render_template("foo", variables)
Mypy gives an error:
Argument 2 to "render_template" has incompatible type "Dict[str, str]"; expected "Dict[str, Union[int, float, str, None]]"
It seems to me that any value of type Dict[str, str] should be safe to pass to a function that expects a parameter of type Dict[str, Union[int, float, str, None]]. Why doesn’t that work by default? Is there anything I can do to make this work?
One minor edit gets this to work:
variables: VariablesDict = {"key": "value"}
The MyPy docs list three solutions to the invariance problem:
- Use an explicit type annotation (that is my answer shown above).
- Make a copy of the right hand side (this is egregious).
- Use immutable collections as annotations whenever possible: (that matches the excellent answer from @Samwise).
FWIW, the MyPy error message in this case is pretty good. It first gives the difference between the inferred type and the expected type, then it refers you to the relevant section of the docs, and lastly, it makes an appropriate suggestion:
tmp12.py:12: error: Argument 2 to "render_template" has incompatible type "Dict[str, str]"; expected "Dict[str, Union[int, float, st
r, None]]"
tmp12.py:12: note: "Dict" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
tmp12.py:12: note: Consider using "Mapping" instead, which is covariant in the value type
The reason it doesn’t work is that Dict is mutable, and a function which accepts a Dict[str, int|float|str|None] could therefore reasonably insert any of those types into its argument. If the argument was actually a Dict[str, str], it now contains values that violate its type. (For more on this, google "covariance/contravariance/invariance" and "Liskov Substitution Principle" — as a general rule, mutable containers are invariant over their generic type[s].)
As long as render_template doesn’t need to modify the dict you pass to it, an easy fix is to have it take a Mapping (which is an abstract supertype of dict that doesn’t imply mutability, and is therefore covariant) instead of a Dict:
def render_template(name: str, variables: Mapping[str, Union[int, float, str, None]]):
...
It seems to me that any value of type
Dict[str, str]should be safe to pass to a function that expects a parameter of typeDict[str, Union[int, float, str, None]].
Unfortunately, that assumption is wrong. Because dict is mutable, it must be invariant in its second type parameter.
Why doesn’t that work by default?
It doesn’t work because it is not type-safe: render_template expects an argument of type dict[str, Union[int, float, str, None]], which means that (among other things), it expects to be able to store an int in the dict.
However, you cannot store an int in a dict[str, str], therefore MyPy rightfully rejects the code.
Is there anything I can do to make this work?
The difference between the two snippets of code is that type inference in MyPy is local. Therefore, in the second example, MyPy does not take into account how the variable variables is used later on. It only infers its type from the assignment expression, and it tries to infer the most strict type possible.
Now, as I mentioned above, the problem is that dict is mutable. Therefore, if you can make it immutable, your code will work.
Thankfully, there is just the type for you: Mapping. Mappings are immutable, therefore, they can be covariant in their second type parameter.