Can the * (unpacking) operator be typed in Python? Or any other variadic args function such that all variadic types are in the result type?

Question

Working with type stubs, I’m wondering if it’s possible to express a type in Python that allows you to type this correctly for any number of arguments:

def test(*args):
  return args

At first glance, I came with:

T = TypeVar('T')
def test(*args: T) -> Tuple[T, ...]:
  return args

But this of course will only type correctly the first T.

Is the only possible way to write the overrides for all arities?

T1 = TypeVar('T1')
T2 = TypeVar('T2')
T3 = TypeVar('T3')
T4 = TypeVar('T4')

@overload
def test(arg1: T1) -> Tuple[T1]: ...
@overload
def test(arg1: T1, arg2: T2) -> Tuple[T1, T2]: ...
@overload
def test(arg1: T1, arg2: T2, arg3: T3) -> Tuple[T1, T2, T3]: ...
@overload
def test(arg1: T1, arg2: T2, arg3: T3, arg4: T4) -> Tuple[T1, T2, T3, T4]: ...
# etc
def test(*args: Any) -> Tuple[Any, ...]:
  return args

This is not complete either, since it does not carry enough type information to type something like:

x: Tuple[int, int, str] = test(*[1, 2, "4"])

Asked By: ssice

||

Source

Answer 1

TLDR: @overload is currently the only viable way to annotate some level of variance. PEP 646 — Variadic Generics will enable annotating variadics in Python 3.11 or via typing_extensions.

*args as a Type Variable Tuple

If *args is annotated as a type variable tuple, however, the types of the individual arguments become the types in the type variable tuple:
Ts = TypeVarTuple('Ts')

def args_to_tuple(*args: *Ts) -> Tuple[*Ts]: ...

args_to_tuple(1, 'a')  # Inferred type is Tuple[int, str]

Note that older versions may not support the *TS syntax and need typing.Unpack instead, which is also available via typing_extensions.

The correct approach to annotate *args is to decide on some level of supported "variance length", and explicitly type this using @overload. Notably, explicit parameters must be positional only – they must be either __anonymous or positional, /. A final catch-all variadic @overload handles the case of more arguments.

from typing import TypeVar, Tuple, Any, overload

T = TypeVar('T')
T1 = TypeVar('T1')
T2 = TypeVar('T2')
T3 = TypeVar('T3')

# positional parameters via `, /` – Python 3.8+ only
@overload
def test(arg1: T1, /) -> Tuple[T1]: ...
# positional parameters via double underscore prefix
@overload
def test(__arg1: T1, __arg2: T2) -> Tuple[T1, T2]: ...
@overload
def test(__arg1: T1, __arg2: T2, __arg3: T3) -> Tuple[T1, T2, T3]: ...
# etc
...
# catch all variadic signature for all other cases
@overload
def test(*args: T) -> Tuple[T, ...]: ...

# implementation can use Any to simplify matching all overloads
def test(*args: Any) -> Tuple[Any, ...]:
  return args

reveal_type(test(1, 2, "three"))     # note: Revealed type is 'Tuple[builtins.int*, builtins.int*, builtins.str*]'
reveal_type(test(1, 2, "three", 4))  # note: Revealed type is 'builtins.tuple[builtins.object*]'
reveal_type(test(1, 2, 3, 4))        # note: Revealed type is 'builtins.tuple[builtins.int*]'

It is worth noting that while packing into variadic parameters can be typed, unpacking arguments generally cannot: Any container but tuple is arbitrary length – for example List[int] = [1, 2, 3] – and thus has no exact type information for its elements.

# unpack tuple of int, int
reveal_type(test(*(1, 2,)))  # note: Revealed type is 'Tuple[builtins.int*, builtins.int*]'
# unpack list of some ints
reveal_type(test(*[1, 2,]))  # note: Revealed type is 'builtins.tuple[builtins.int*]'

Answered By: MisterMiyagi

Answer 2

This can solved through TypeVarTuple from PEP646, implemented in Python 3.11 or forward compat module typing-extensions.

See this analog example:

from __future__ import annotations

from typing import Any, Callable, Generic
from typing_extensions import TypeVarTuple, Unpack  # typing_extensions only needed for Python < 3.11

Ts = TypeVarTuple("Ts")

class Signal(Generic[Unpack[Ts]]):
    def add_callback(self, func: Callable[[Unpack[Ts]], Any]) -> None:
        ...

    def emit(self, *args: Unpack[Ts]) -> None:
        ...

def callback(a: int, b: str) -> None:
    ...

def callback_bad(a: str) -> None:
    ...

sig: Signal[int, str] = Signal()
sig.add_callback(callback)  # Good lint
sig.add_callback(callback_bad)  # Bad lint

sig.emit(1223, "foo")  # Good lint
sig.emit("bar")  # Bad lint

Also see Dynamic TypeVar for a sequence of types for a solution.

Answered By: n1nj4

Can the * (unpacking) operator be typed in Python? Or any other variadic args function such that all variadic types are in the result type?

Question:

Answers:

`*args` as a Type Variable Tuple

Can the * (unpacking) operator be typed in Python? Or any other variadic args function such that all variadic types are in the result type?

Question:

Answers:

*args as a Type Variable Tuple

`*args` as a Type Variable Tuple