When is d1==d2 not equivalent to d1.__eq__(d2)?
Question:
According to the docs (in Python 3.8):
By default, object
implements __eq__()
by using is
, returning NotImplemented
in the case of a false comparison: True if x is y else NotImplemented
.
And also:
The correspondence between operator symbols and method names is as follows: […] x==y
calls x.__eq__(y)
So I expect
==
to be equivalent to __eq__()
and
- a custom class without an explicitly defined
__eq__
to return NotImplemented
when using ==
to compare two different instances of the class. Yet in the following, ==
comparison returns False
, while __eq__()
returns NotImplemented
:
class Dummy():
def __init__(self, a):
self.a = a
d1 = Dummy(3)
d2 = Dummy(3)
d1 == d2 # False
d1.__eq__(d2) # NotImplemented
Why?
Answers:
The reason is that if one side of the operation cannot (or will not) provide an answer, the other side is allowed a say in handling this comparison. A common example is float/int comparisons:
>>> 1 == 1.0
True
>>> (1).__eq__(1.0)
NotImplemented
>>> (1.0).__eq__(1)
True
With int
and float
, neither is a subclass of the other, and an int
doesn’t have anything to say about whether it’s equal to some float
or not. The equality comparison gets handled by the float
type.
It’s easier to understand if you have distinct types for the left and right sides, and add some debug printout in the hooks:
class Left:
def __eq__(self, other):
print("trying Left.__eq__")
return NotImplemented
class Right:
def __eq__(self, other):
print("trying Right.__eq__")
return True
Demo:
>>> d1 = Left()
>>> d2 = Right()
>>> d1.__eq__(d2)
trying Left.__eq__
NotImplemented
>>> d2.__eq__(d1)
trying Right.__eq__
True
>>> d1 == d2 # Left.__eq__ "opts out" of the comparison, Python asks other side
trying Left.__eq__
trying Right.__eq__
True
>>> d2 == d1 # Right.__eq__ returns a result first, Left.__eq__ isn't considered
trying Right.__eq__
True
Left side type(d1).__eq__
opts out by returning NotImplemented
, which allows the right hand side a "second chance" at handling the operation. If left side had returned False
instead of returning NotImplemented
, Python wouldn’t attempt the right side at all and the result would of d1 == d2
be False
. If both sides return NotImplemented
, like in your Dummy
example, then the objects are considered unequal unless they are identical (i.e. same instance).
The documentation here is not very clear and definitely needs to be improved.
When a Python "internal" method returns NotImplemented
, it usually means "find some other way to do this operation". What Python does when it can’t find "some other way …." is dependent on the operation, and less well documented than it should be. So for example, if x.__lt__(y)
returns NotImplemented
, Python might try calling y.__gt__(x)
.
In the case of ==
, Python returns False
if x.__eq__(y)
and y.__eq__(x)
both return NotImplemented
. This should be documented better.
==
calls the right operand’s .__eq__()
if the left operand’s .__eq__()
returns NotImplemented
and if both sides return NotImplemented
, ==
will return false.
You can see this behavior by changing your class:
class Dummy():
def __eq__(self, o):
print(f'{id(self)} eq')
return NotImplemented
d1 = Dummy()
d2 = Dummy()
print(d1 == d2)
# 2180936917136 eq
# 2180936917200 eq
# False
This is a common behavior for operators where Python test if the left operand have an implementation, if not, Python call the same or reflection (e.g. if x.__lt__(y)
is not implemented y.__gt__(x)
is called) operator of the right operand.
class Dummy:
def __eq__(self, o):
print(f"{id(self)} eq")
return NotImplemented
def __lt__(self, o):
"""reflexion of gt"""
print(f"{id(self)} lt")
return NotImplemented
def __gt__(self, o):
"""reflexion of lt"""
print(f"{id(self)} gt")
return False
def __le__(self, o):
"""reflexion of ge"""
print(f"{id(self)} le")
return NotImplemented
def __ge__(self, o):
"""reflexion of le"""
print(f"{id(self)} ge")
return False
d1 = Dummy()
d2 = Dummy()
print(d1 == d2)
# 2480053379984 eq
# 2480053380688 eq
# False
print(d1 < d2)
# 2480053379984 lt
# 2480053380688 gt
# False
print(d1 <= d2)
# 2480053379984 le
# 2480053380688 ge
# False
! Exception to the left before right:
If the operands are of different types, and right operand’s type is a direct or indirect subclass of the left operand’s type, the reflected method of the right operand has priority, otherwise the left operand’s method has priority.
According to the docs (in Python 3.8):
By default,
object
implements__eq__()
by usingis
, returningNotImplemented
in the case of a false comparison:True if x is y else NotImplemented
.
And also:
The correspondence between operator symbols and method names is as follows: […]
x==y
callsx.__eq__(y)
So I expect
==
to be equivalent to__eq__()
and- a custom class without an explicitly defined
__eq__
to returnNotImplemented
when using==
to compare two different instances of the class. Yet in the following,==
comparison returnsFalse
, while__eq__()
returnsNotImplemented
:
class Dummy():
def __init__(self, a):
self.a = a
d1 = Dummy(3)
d2 = Dummy(3)
d1 == d2 # False
d1.__eq__(d2) # NotImplemented
Why?
The reason is that if one side of the operation cannot (or will not) provide an answer, the other side is allowed a say in handling this comparison. A common example is float/int comparisons:
>>> 1 == 1.0
True
>>> (1).__eq__(1.0)
NotImplemented
>>> (1.0).__eq__(1)
True
With int
and float
, neither is a subclass of the other, and an int
doesn’t have anything to say about whether it’s equal to some float
or not. The equality comparison gets handled by the float
type.
It’s easier to understand if you have distinct types for the left and right sides, and add some debug printout in the hooks:
class Left:
def __eq__(self, other):
print("trying Left.__eq__")
return NotImplemented
class Right:
def __eq__(self, other):
print("trying Right.__eq__")
return True
Demo:
>>> d1 = Left()
>>> d2 = Right()
>>> d1.__eq__(d2)
trying Left.__eq__
NotImplemented
>>> d2.__eq__(d1)
trying Right.__eq__
True
>>> d1 == d2 # Left.__eq__ "opts out" of the comparison, Python asks other side
trying Left.__eq__
trying Right.__eq__
True
>>> d2 == d1 # Right.__eq__ returns a result first, Left.__eq__ isn't considered
trying Right.__eq__
True
Left side type(d1).__eq__
opts out by returning NotImplemented
, which allows the right hand side a "second chance" at handling the operation. If left side had returned False
instead of returning NotImplemented
, Python wouldn’t attempt the right side at all and the result would of d1 == d2
be False
. If both sides return NotImplemented
, like in your Dummy
example, then the objects are considered unequal unless they are identical (i.e. same instance).
The documentation here is not very clear and definitely needs to be improved.
When a Python "internal" method returns NotImplemented
, it usually means "find some other way to do this operation". What Python does when it can’t find "some other way …." is dependent on the operation, and less well documented than it should be. So for example, if x.__lt__(y)
returns NotImplemented
, Python might try calling y.__gt__(x)
.
In the case of ==
, Python returns False
if x.__eq__(y)
and y.__eq__(x)
both return NotImplemented
. This should be documented better.
==
calls the right operand’s .__eq__()
if the left operand’s .__eq__()
returns NotImplemented
and if both sides return NotImplemented
, ==
will return false.
You can see this behavior by changing your class:
class Dummy():
def __eq__(self, o):
print(f'{id(self)} eq')
return NotImplemented
d1 = Dummy()
d2 = Dummy()
print(d1 == d2)
# 2180936917136 eq
# 2180936917200 eq
# False
This is a common behavior for operators where Python test if the left operand have an implementation, if not, Python call the same or reflection (e.g. if x.__lt__(y)
is not implemented y.__gt__(x)
is called) operator of the right operand.
class Dummy:
def __eq__(self, o):
print(f"{id(self)} eq")
return NotImplemented
def __lt__(self, o):
"""reflexion of gt"""
print(f"{id(self)} lt")
return NotImplemented
def __gt__(self, o):
"""reflexion of lt"""
print(f"{id(self)} gt")
return False
def __le__(self, o):
"""reflexion of ge"""
print(f"{id(self)} le")
return NotImplemented
def __ge__(self, o):
"""reflexion of le"""
print(f"{id(self)} ge")
return False
d1 = Dummy()
d2 = Dummy()
print(d1 == d2)
# 2480053379984 eq
# 2480053380688 eq
# False
print(d1 < d2)
# 2480053379984 lt
# 2480053380688 gt
# False
print(d1 <= d2)
# 2480053379984 le
# 2480053380688 ge
# False
! Exception to the left before right:
If the operands are of different types, and right operand’s type is a direct or indirect subclass of the left operand’s type, the reflected method of the right operand has priority, otherwise the left operand’s method has priority.