How to type annotate reduce function?

Question

I’m writing a very thin wrapper on top of list and I want to define a method called reduce, but I’m struggling to annotate it properly such that pylance, mypy & pylint cut their complaints whenever I use the method, or even define it.

I was perturbed to realize that almost None of Python’s builtin libraries are type annotated.

This is my implementation attempt:

    def reduce(self, func: Callable[[list[T], list[T]], list[T]] = lambda x, y: x + y, default: Optional[T] = None) -> 'List[T]':  # type: ignore
        from functools import reduce
        if default is None: return List(reduce(func, self.list))  # type: ignore
        return List(reduce(func, self.list, default))  # type: ignore

This fails when my List is actually a list of strings

a: List[str] = List(['a', 'b'])
b = a.reduce(lambda x, y: x + y)

Obviously here, the type checkers and linters say they expect list[T] while I passed str.

Asked By: Alex

||

Source

Answer 1

I think in this situation one would use a generic type, otherwise it can’t reason about your types fully. Heres an example of a list using a Generic type which supports a Protocol that allows it to be reduced:

from functools import reduce
from typing import Sequence, TypeVar, Callable, Generic, Dict, Protocol, cast

A = TypeVar("A")


# define that our type supports the add operator, since it needs to be able to to reduce it
class SupportsAdd(Protocol[A]):
    def __add__(self, __other: A) -> A:
        ...


T = TypeVar("T", bound=SupportsAdd)


def _reduce_default(x: T, y: T) -> T:
    return x + y


# instead of using None as the default, we use a special object
# so that we can distinguish between None and a missing default
#
# (the underlying reduce function does the same thing)
_missing_default = object()


class MyList(Generic[T]):
    def __init__(self, initialdata: Sequence[T]) -> None:
        self.data = list(initialdata)

    def __getitem__(self, i: int) -> T:
        return self.data[i]

    def __setitem__(self, i: int, v: T) -> None:
        self.data[i] = v

    def __len__(self) -> int:
        return len(self.data)

    def __delitem__(self, i: int) -> None:
        del self.data[i]

    def reduce(
        self,
        func: Callable[[T, T], T] = _reduce_default,
        default: T = cast(T, _missing_default),
    ) -> T:
        if default is _missing_default:
            return reduce(_reduce_default, self.data)
        else:
            return reduce(func, self.data, default)

    # just another method that uses the types
    def add_keys(self, keys: Dict[T, int]) -> None:
        for k in keys:
            self.data.append(k)

Some usage of it:

def test() -> None:
    ll: MyList[int] = MyList([1, 2, 3])

    print(ll[1])

    ll[1] = 5

    print(ll[1])

    # add some keys
    ll.add_keys({30: 1, 50: 2, 60: 3})

    print(len(ll))

    val: int  # define the type separately, mypy warns us if it doesnt match
    val = ll.reduce()

    print(val)

    # using strings instead
    ll2: MyList[str] = MyList(["a", "b", "c"])

    print(ll2.reduce())

2
5
6
149
abc

If you tried to use some class which didn’t support __add__, mypy would warn you:

# test class to see if classes without __add__ fail
class NoAdd:
    def __init__(self, v: int) -> None:
        self.v = v

x: MyList[NoAdd] = [NoAdd(1)]

test.py:100: error: Type argument "NoAdd" of "MyList" must be a subtype of "SupportsAdd[Any]"  [type-var]
test.py:100: error: Value of type variable "T" of "MyList" cannot be "NoAdd"  [type-var]
Found 2 errors in 1 file (checked 1 source file)

Answered By: Sean Breckenridge

How to type annotate reduce function?

Question:

Answers: