How should I use the Optional type hint?
Question:
I’m trying to understand how to use the Optional type hint. From PEP-484, I know I can use Optional for def test(a: int = None) either as def test(a: Union[int, None]) or def test(a: Optional[int]).
But how about following examples?
def test(a : dict = None):
#print(a) ==> {'a': 1234}
#or
#print(a) ==> None
def test(a : list = None):
#print(a) ==> [1,2,3,4, 'a', 'b']
#or
#print(a) ==> None
If Optional[type] seems to mean the same thing as Union[type, None], why should I use Optional[] at all?
Answers:
Optional[...] is a shorthand notation for Union[..., None], telling the type checker that either an object of the specific type is required, or None is required. ... stands for any valid type hint, including complex compound types or a Union[] of more types. Whenever you have a keyword argument with default value None, you should use Optional. (Note: If you are targeting Python 3.10 or newer, PEP 604 introduced a better syntax, see below).
So for your two examples, you have dict and list container types, but the default value for the a keyword argument shows that None is permitted too so use Optional[...]:
from typing import Optional
def test(a: Optional[dict] = None) -> None:
#print(a) ==> {'a': 1234}
#or
#print(a) ==> None
def test(a: Optional[list] = None) -> None:
#print(a) ==> [1, 2, 3, 4, 'a', 'b']
#or
#print(a) ==> None
There is technically no difference between using Optional[] on a Union[], or just adding None to the Union[]. So Optional[Union[str, int]] and Union[str, int, None] are exactly the same thing.
Personally, I’d stick with always using Optional[] when setting the type for a keyword argument that uses = None to set a default value, this documents the reason why None is allowed better. Moreover, it makes it easier to move the Union[...] part into a separate type alias, or to later remove the Optional[...] part if an argument becomes mandatory.
For example, say you have
from typing import Optional, Union
def api_function(optional_argument: Optional[Union[str, int]] = None) -> None:
"""Frob the fooznar.
If optional_argument is given, it must be an id of the fooznar subwidget
to filter on. The id should be a string, or for backwards compatibility,
an integer is also accepted.
"""
then documentation is improved by pulling out the Union[str, int] into a type alias:
from typing import Optional, Union
# subwidget ids used to be integers, now they are strings. Support both.
SubWidgetId = Union[str, int]
def api_function(optional_argument: Optional[SubWidgetId] = None) -> None:
"""Frob the fooznar.
If optional_argument is given, it must be an id of the fooznar subwidget
to filter on. The id should be a string, or for backwards compatibility,
an integer is also accepted.
"""
The refactor to move the Union[] into an alias was made all the much easier because Optional[...] was used instead of Union[str, int, None]. The None value is not a ‘subwidget id’ after all, it’s not part of the value, None is meant to flag the absence of a value.
Side note: Unless your code only has to support Python 3.9 or newer, you want to avoid using the standard library container types in type hinting, as you can’t say anything about what types they must contain. So instead of dict and list, use typing.Dict and typing.List, respectively. And when only reading from a container type, you may just as well accept any immutable abstract container type; lists and tuples are Sequence objects, while dict is a Mapping type:
from typing import Mapping, Optional, Sequence, Union
def test(a: Optional[Mapping[str, int]] = None) -> None:
"""accepts an optional map with string keys and integer values"""
# print(a) ==> {'a': 1234}
# or
# print(a) ==> None
def test(a: Optional[Sequence[Union[int, str]]] = None) -> None:
"""accepts an optional sequence of integers and strings
# print(a) ==> [1, 2, 3, 4, 'a', 'b']
# or
# print(a) ==> None
In Python 3.9 and up, the standard container types have all been updated to support using them in type hints, see PEP 585. But, while you now can use dict[str, int] or list[Union[int, str]], you still may want to use the more expressive Mapping and Sequence annotations to indicate that a function won’t be mutating the contents (they are treated as ‘read only’), and that the functions would work with any object that works as a mapping or sequence, respectively.
Python 3.10 introduces the | union operator into type hinting, see PEP 604. Instead of Union[str, int] you can write str | int. In line with other type-hinted languages, the preferred (and more concise) way to denote an optional argument in Python 3.10 and up, is now Type | None, e.g. str | None or list | None.
Directly from mypy typing module docs.
Optional[str] is just a shorthand or alias for Union[str, None]. It exists mostly as a convenience to help function signatures look a little cleaner.
Update for Python 3.10+
you can now use the pipe operator as well.
# Python < 3.10
def get_cars(size: Optional[str]=None):
pass
# Python 3.10+
def get_cars(size: str|None=None):
pass
While the accepted answer is the correct answer, one additional thing to note is that, in the context of kwargs, both Optional[...] and Union[..., None] are redundant and unnecessary. If you’re immediately setting your kwarg to None, then both mypy and IDEs assume the obvious and automatically treat the arg as Optional[...].
IDE:
mypy:
For variables and method/function return values, Optional[...] is still necessary, however, as mypy cannot know, in those cases, to automatically assume anything.
Up until Python 3.9 if you wanted to hint for a nullable value you had two options:
import typing
def foo(bar: typing.Optional[str]):
....
def foo(bar: typing.Union[str, None]):
....
From Python 3.9 you are not required to use typing module:
def foo(bar: str = None):
....
Note that since Python 3.10 you can simplify your code and type it like:
def foo(
bar: int | None = None,
another_bar: Callable[[int, list, float, datetime | None], str],
):
As already noted in several of the comments, ever since Python 3.7 it has been possible to use the new-style type annotations via a __future__ import:
from __future__ import annotations
def test(a: dict[str, int] | None) -> None:
...
As for automatically answering this and many other general best-practices questions, I would highly recommend using pyupgrade to automatically reformat your type annotations and the rest of your code with modern Python style. For a single file, after adding the __future__ import, run pyupgrade --py37-plus --keep-runtime-typing file.py.
If you use Git, then pyupgrade can be set up as a pre-commit hook so that your code always stays modern. Here is the block I use:
# Upgrade code style to the specified minimum supported Python version.
- repo: https://github.com/asottile/pyupgrade
rev: v3.3.1 # Adjust this to the latest version of pyupgrade
hooks:
- id: pyupgrade
# Adjust the following to the minimum supported Python version for your project.
args:
- --py37-plus
- --keep-runtime-typing
Note: The --keep-runtime-typing argument is required in case you use Pydantic, FastAPI, or other similar tools which rely on runtime typing. Otherwise this argument can be safely omitted.
I’m trying to understand how to use the Optional type hint. From PEP-484, I know I can use Optional for def test(a: int = None) either as def test(a: Union[int, None]) or def test(a: Optional[int]).
But how about following examples?
def test(a : dict = None):
#print(a) ==> {'a': 1234}
#or
#print(a) ==> None
def test(a : list = None):
#print(a) ==> [1,2,3,4, 'a', 'b']
#or
#print(a) ==> None
If Optional[type] seems to mean the same thing as Union[type, None], why should I use Optional[] at all?
Optional[...] is a shorthand notation for Union[..., None], telling the type checker that either an object of the specific type is required, or None is required. ... stands for any valid type hint, including complex compound types or a Union[] of more types. Whenever you have a keyword argument with default value None, you should use Optional. (Note: If you are targeting Python 3.10 or newer, PEP 604 introduced a better syntax, see below).
So for your two examples, you have dict and list container types, but the default value for the a keyword argument shows that None is permitted too so use Optional[...]:
from typing import Optional
def test(a: Optional[dict] = None) -> None:
#print(a) ==> {'a': 1234}
#or
#print(a) ==> None
def test(a: Optional[list] = None) -> None:
#print(a) ==> [1, 2, 3, 4, 'a', 'b']
#or
#print(a) ==> None
There is technically no difference between using Optional[] on a Union[], or just adding None to the Union[]. So Optional[Union[str, int]] and Union[str, int, None] are exactly the same thing.
Personally, I’d stick with always using Optional[] when setting the type for a keyword argument that uses = None to set a default value, this documents the reason why None is allowed better. Moreover, it makes it easier to move the Union[...] part into a separate type alias, or to later remove the Optional[...] part if an argument becomes mandatory.
For example, say you have
from typing import Optional, Union
def api_function(optional_argument: Optional[Union[str, int]] = None) -> None:
"""Frob the fooznar.
If optional_argument is given, it must be an id of the fooznar subwidget
to filter on. The id should be a string, or for backwards compatibility,
an integer is also accepted.
"""
then documentation is improved by pulling out the Union[str, int] into a type alias:
from typing import Optional, Union
# subwidget ids used to be integers, now they are strings. Support both.
SubWidgetId = Union[str, int]
def api_function(optional_argument: Optional[SubWidgetId] = None) -> None:
"""Frob the fooznar.
If optional_argument is given, it must be an id of the fooznar subwidget
to filter on. The id should be a string, or for backwards compatibility,
an integer is also accepted.
"""
The refactor to move the Union[] into an alias was made all the much easier because Optional[...] was used instead of Union[str, int, None]. The None value is not a ‘subwidget id’ after all, it’s not part of the value, None is meant to flag the absence of a value.
Side note: Unless your code only has to support Python 3.9 or newer, you want to avoid using the standard library container types in type hinting, as you can’t say anything about what types they must contain. So instead of dict and list, use typing.Dict and typing.List, respectively. And when only reading from a container type, you may just as well accept any immutable abstract container type; lists and tuples are Sequence objects, while dict is a Mapping type:
from typing import Mapping, Optional, Sequence, Union
def test(a: Optional[Mapping[str, int]] = None) -> None:
"""accepts an optional map with string keys and integer values"""
# print(a) ==> {'a': 1234}
# or
# print(a) ==> None
def test(a: Optional[Sequence[Union[int, str]]] = None) -> None:
"""accepts an optional sequence of integers and strings
# print(a) ==> [1, 2, 3, 4, 'a', 'b']
# or
# print(a) ==> None
In Python 3.9 and up, the standard container types have all been updated to support using them in type hints, see PEP 585. But, while you now can use dict[str, int] or list[Union[int, str]], you still may want to use the more expressive Mapping and Sequence annotations to indicate that a function won’t be mutating the contents (they are treated as ‘read only’), and that the functions would work with any object that works as a mapping or sequence, respectively.
Python 3.10 introduces the | union operator into type hinting, see PEP 604. Instead of Union[str, int] you can write str | int. In line with other type-hinted languages, the preferred (and more concise) way to denote an optional argument in Python 3.10 and up, is now Type | None, e.g. str | None or list | None.
Directly from mypy typing module docs.
Optional[str]is just a shorthand or alias forUnion[str, None]. It exists mostly as a convenience to help function signatures look a little cleaner.
Update for Python 3.10+
you can now use the pipe operator as well.
# Python < 3.10
def get_cars(size: Optional[str]=None):
pass
# Python 3.10+
def get_cars(size: str|None=None):
pass
While the accepted answer is the correct answer, one additional thing to note is that, in the context of kwargs, both Optional[...] and Union[..., None] are redundant and unnecessary. If you’re immediately setting your kwarg to None, then both mypy and IDEs assume the obvious and automatically treat the arg as Optional[...].
IDE:
mypy:
For variables and method/function return values, Optional[...] is still necessary, however, as mypy cannot know, in those cases, to automatically assume anything.
Up until Python 3.9 if you wanted to hint for a nullable value you had two options:
import typing
def foo(bar: typing.Optional[str]):
....
def foo(bar: typing.Union[str, None]):
....
From Python 3.9 you are not required to use typing module:
def foo(bar: str = None):
....
Note that since Python 3.10 you can simplify your code and type it like:
def foo(
bar: int | None = None,
another_bar: Callable[[int, list, float, datetime | None], str],
):
As already noted in several of the comments, ever since Python 3.7 it has been possible to use the new-style type annotations via a __future__ import:
from __future__ import annotations
def test(a: dict[str, int] | None) -> None:
...
As for automatically answering this and many other general best-practices questions, I would highly recommend using pyupgrade to automatically reformat your type annotations and the rest of your code with modern Python style. For a single file, after adding the __future__ import, run pyupgrade --py37-plus --keep-runtime-typing file.py.
If you use Git, then pyupgrade can be set up as a pre-commit hook so that your code always stays modern. Here is the block I use:
# Upgrade code style to the specified minimum supported Python version.
- repo: https://github.com/asottile/pyupgrade
rev: v3.3.1 # Adjust this to the latest version of pyupgrade
hooks:
- id: pyupgrade
# Adjust the following to the minimum supported Python version for your project.
args:
- --py37-plus
- --keep-runtime-typing
Note: The --keep-runtime-typing argument is required in case you use Pydantic, FastAPI, or other similar tools which rely on runtime typing. Otherwise this argument can be safely omitted.