Why does python `any` return a bool instead of the value?
Question:
and and or return the last element they evaluated, but why doesn’t Python’s built-in function any?
I mean it’s pretty easy to implement oneself like this, but I’m still left wondering why.
def any(l):
for x in l:
if x:
return x
return x
edit:
To add to the answers below, here’s an actual quote from that same mailing list of ye mighty emperor on the issue:
Whether to always return True and False or the first faling / passing
element? I played with that too before blogging, and realized that the
end case (if the sequence is empty or if all elements fail the test)
can never be made to work satisfactory: picking None feels weird if
the argument is an iterable of bools, and picking False feels weird if
the argument is an iterable of non-bool objects.
Guido van Rossum (home page: http://www.python.org/~guido/)
Answers:
Any returns a boolean because it effectively treats its argument as a list of bools before considering if any of them are true. It is returning the element it evaluates, but this happens to be a bool.
When would you want to use your version of any? If it’s on a list of bools then you already have the correct answer. Otherwise you are just guarding against None and might be expressed as:
filter(lambda x: x != None, l)[0]
or:
[x for x in l if x != None][0]
Which is a clearer statement of intent.
It’s not immediately obvious that any‘s value could be either False or one of the values in the input. Also, most uses would look like
tmp = any(iterable)
if tmp:
tmp.doSomething()
else:
raise ValueError('Did not find anything')
That’s Look Before You Leap and therefore unpythonic. Compare to:
next(i for i in iterable if i).doSomething()
# raises StopIteration if no value is true
The behavior of and and or was historically useful as a drop-in for the then-unavailable conditional expression.
I asked this same question on python-ideas, and was told the reason was that any() and all() need to return a value when the sequence is empty, and those values must be False and True. This seems like a weak argument to me.
The functions can’t change now, but I think they would be more useful, and better analogs of the and and or operators they generalize, if they returned the first true-ish or false-ish value they encountered.
The behavior of and and or exists for historical reasons.
Before Python had a ternary operation / conditional expression, you used and and or if you wanted to use a value on a condition. Any such expression can be rewritten with the conditional expression syntax:
true_val if condition else false_val
Essentially, they are overloaded with two functions, and for compatibility reasons, they haven’t been changed.
That is not a reason to overload other operations. any seems like it should tell you whether or not a condition is true for any item, which is a boolean, so it should return a bool.
and and or can be sensibly defined in a way that they always return one of their operands. However, any and all cannot sensibly be defined always to return a value from their input sequence: specifically they cannot do so when the list is empty. Both any and all currently have a well defined result in this situation: any returns False and all returns True. You would be forced to sometimes return a boolean value and sometimes return an item from the sequence, which makes for an unpleasant and surprising interface. Much better to be simple and consistent.
This very issue came up up on the Python developer’s mailing list in 2005, when Guido Van Rossum proposed adding any and all to Python 2.5.
Bill Janssen requested that they be implemented as
def any(S):
for x in S:
if x:
return x
return S[-1]
def all(S):
for x in S:
if not x:
return x
return S[-1]
Raymond Hettinger, who implemented any and all, responded specifically addressing why any and all don’t act like and and or:
Over time, I’ve gotten feedback about these and other itertools recipes.
No one has objected to the True/False return values in those recipes or
in Guido’s version.
Guido’s version matches the normal expectation of any/all being a
predicate. Also, it avoids the kind of errors/confusion that people
currently experience with Python’s unique implementation of “and” and
“or”.
Returning the last element is not evil; it’s just weird, unexpected, and
non-obvious. Resist the urge to get tricky with this one.
The mailing list largely concurred, leaving the implementation as you see it today.
Starting Python 3.8, and the introduction of assignment expressions (PEP 572) (:= operator), we can alternatively explicitly capture a witness of an any expression or a counterexample of an all expression:
To quote a couple examples from the PEP description:
if any(len(long_line := line) >= 100 for line in lines):
print("Extremely long line:", long_line)
if all((nonblank := line).strip() == '' for line in lines):
print("All lines are blank")
else:
print("First non-blank line:", nonblank)
and and or return the last element they evaluated, but why doesn’t Python’s built-in function any?
I mean it’s pretty easy to implement oneself like this, but I’m still left wondering why.
def any(l):
for x in l:
if x:
return x
return x
edit:
To add to the answers below, here’s an actual quote from that same mailing list of ye mighty emperor on the issue:
Whether to always return True and False or the first faling / passing
element? I played with that too before blogging, and realized that the
end case (if the sequence is empty or if all elements fail the test)
can never be made to work satisfactory: picking None feels weird if
the argument is an iterable of bools, and picking False feels weird if
the argument is an iterable of non-bool objects.Guido van Rossum (home page: http://www.python.org/~guido/)
Any returns a boolean because it effectively treats its argument as a list of bools before considering if any of them are true. It is returning the element it evaluates, but this happens to be a bool.
When would you want to use your version of any? If it’s on a list of bools then you already have the correct answer. Otherwise you are just guarding against None and might be expressed as:
filter(lambda x: x != None, l)[0]
or:
[x for x in l if x != None][0]
Which is a clearer statement of intent.
It’s not immediately obvious that any‘s value could be either False or one of the values in the input. Also, most uses would look like
tmp = any(iterable)
if tmp:
tmp.doSomething()
else:
raise ValueError('Did not find anything')
That’s Look Before You Leap and therefore unpythonic. Compare to:
next(i for i in iterable if i).doSomething()
# raises StopIteration if no value is true
The behavior of and and or was historically useful as a drop-in for the then-unavailable conditional expression.
I asked this same question on python-ideas, and was told the reason was that any() and all() need to return a value when the sequence is empty, and those values must be False and True. This seems like a weak argument to me.
The functions can’t change now, but I think they would be more useful, and better analogs of the and and or operators they generalize, if they returned the first true-ish or false-ish value they encountered.
The behavior of and and or exists for historical reasons.
Before Python had a ternary operation / conditional expression, you used and and or if you wanted to use a value on a condition. Any such expression can be rewritten with the conditional expression syntax:
true_val if condition else false_val
Essentially, they are overloaded with two functions, and for compatibility reasons, they haven’t been changed.
That is not a reason to overload other operations. any seems like it should tell you whether or not a condition is true for any item, which is a boolean, so it should return a bool.
and and or can be sensibly defined in a way that they always return one of their operands. However, any and all cannot sensibly be defined always to return a value from their input sequence: specifically they cannot do so when the list is empty. Both any and all currently have a well defined result in this situation: any returns False and all returns True. You would be forced to sometimes return a boolean value and sometimes return an item from the sequence, which makes for an unpleasant and surprising interface. Much better to be simple and consistent.
This very issue came up up on the Python developer’s mailing list in 2005, when Guido Van Rossum proposed adding any and all to Python 2.5.
Bill Janssen requested that they be implemented as
def any(S):
for x in S:
if x:
return x
return S[-1]
def all(S):
for x in S:
if not x:
return x
return S[-1]
Raymond Hettinger, who implemented any and all, responded specifically addressing why any and all don’t act like and and or:
Over time, I’ve gotten feedback about these and other itertools recipes.
No one has objected to the True/False return values in those recipes or
in Guido’s version.Guido’s version matches the normal expectation of any/all being a
predicate. Also, it avoids the kind of errors/confusion that people
currently experience with Python’s unique implementation of “and” and
“or”.Returning the last element is not evil; it’s just weird, unexpected, and
non-obvious. Resist the urge to get tricky with this one.
The mailing list largely concurred, leaving the implementation as you see it today.
Starting Python 3.8, and the introduction of assignment expressions (PEP 572) (:= operator), we can alternatively explicitly capture a witness of an any expression or a counterexample of an all expression:
To quote a couple examples from the PEP description:
if any(len(long_line := line) >= 100 for line in lines):
print("Extremely long line:", long_line)
if all((nonblank := line).strip() == '' for line in lines):
print("All lines are blank")
else:
print("First non-blank line:", nonblank)