Why is "dict[int, int]" incompatible with "dict[int, int | str]"?

Question:

import typing

a: dict[int, int] = {}
b: dict[int, int | str] = a
c: typing.Mapping[int, int | str] = a
d: typing.Mapping[int | str, int] = a

Pylance reports an error for b: dict[int, int | str] = a:

Expression of type "dict[int, int]" is incompatible with declared type "dict[int, int | str]"
  "dict[int, int]" is incompatible with "dict[int, int | str]"
    Type parameter "_VT@dict" is invariant, but "int" is not the same as "int | str"
    Consider switching from "dict" to "Mapping" which is covariant in the value type

But c: typing.Mapping[int, int | str] = a is OK.

Additionally, d: typing.Mapping[int | str, int] = a also gets an error:

Expression of type "dict[int, int]" is incompatible with declared type "Mapping[int | str, int]"
  "dict[int, int]" is incompatible with "Mapping[int | str, int]"
    Type parameter "_KT@Mapping" is invariant, but "int" is not the same as "int | str"

Why are these types hint incompatible?
If a function declares a parameter of type dict[int, int | str], how can I pass a dict[int, int] object as its parameter?

Asked By: keakon

||

Answers:

dict type was designed to be completely invariant on key and value. Hence when you assign dict[int, int] to dict[int, int | str], you make the type system raise errors. [1]

Mapping type on the other hand wasn’t designed to be completely invariant but rather is invariant on key and covariant on value. Hence you can assign one Mapping type (dict[int, int]) to another (Mapping[int, int | str]) if they are both covariant on value. if they are invariant on key, you can assign them else you cannot. Hence when you assign dict[int, int] to Mapping[int | str, int], you make the type system raise errors. [2][3]

There is a good reason for the above design in the type system and I will give a few:

1. dict type is a concrete type so it will actually get used in a program.

2. Because of the above mentioned, it was designed the way it was to avoid things like this:

a: dict[int, int] = {}
b: dict[int, int | str] = a
b[0] = 0xDEADBEAF
b[1] = "Bull"

dicts are assigned by reference [4] hence any mutation to b is actually a mutation to a. So if one reads a as follows:

x: int = a[0]
assert isinstance(x, int)
y: int = a[1]
assert isinstance(y, int)

One gets unexpected results. x passes but y doesn’t. It then seems like the type system is contradicting itself. This can cause worse problems in a program.

For posterity, to correctly type a dictionary in Python, use Mapping type to denote a readonly dictionary and use MutableMapping type to denote a read-write dictionary.


[1] Of course Python’s type system doesn’t influence program’s running behaviour but at least linters have some use of this.

[2] dict type is a Mapping type but Mapping type is not a dict type.

[3] Keep in mind that the ordering of types is important in type theory.

[4] All variable names in Python are references to values.

Answered By: Chukwujiobi Canon

This code may seem correct on first sight if you think of only reading from the dicts:

a: dict[int, int] = {}
b: dict[int, int | str] = a

However, if you ever write to them, you can see how it would be wrong to allow that:

b[1] = "x"
assert isinstance(a[1], int)  # fails

The difference with Mapping type is that it does not support modifications.

(Chukwujiobi’s answer explains it well in legal language, if you want a more precise explanation)

Answered By: zvone

The type hint incompatibility you’re seeing is due to the difference between invariance and covariance in Python’s type system.

Invariance: A type variable is said to be invariant if it does not allow subtyping or supertyping. In your case, dict is invariant, which means a dict[int, int] is not considered a subtype of dict[int, int | str]. This is why you’re seeing an error when you try to assign a to b.
Covariance: A type variable is said to be covariant if it allows subtyping. typing.Mapping is covariant in its value type, which means a Mapping[int, int] is considered a subtype of Mapping[int, int | str]. This is why you’re not seeing an error when you assign a to c.
However, typing.Mapping is invariant in its key type, which is why you’re seeing an error when you try to assign a to d.

If a function declares a parameter of type dict[int, int | str], you cannot pass a dict[int, int] object as its parameter due to the invariance of dict. Instead, you should declare the function parameter as typing.Mapping[int, int | str], which is covariant in its value type and will accept a dict[int, int] as an argument. Here’s an example:

def func(param: typing.Mapping[int, int | str]):
    pass

func(a)  

This way, you can pass a dict[int, int] to a function expecting a Mapping[int, int | str]. I hope this helps!

Answered By: Austin H