__lt__ instead of __cmp__
Question:
Python 2.x has two ways to overload comparison operators, __cmp__ or the “rich comparison operators” such as __lt__. The rich comparison overloads are said to be preferred, but why is this so?
Rich comparison operators are simpler to implement each, but you must implement several of them with nearly identical logic. However, if you can use the builtin cmp and tuple ordering, then __cmp__ gets quite simple and fulfills all the comparisons:
class A(object):
def __init__(self, name, age, other):
self.name = name
self.age = age
self.other = other
def __cmp__(self, other):
assert isinstance(other, A) # assumption for this example
return cmp((self.name, self.age, self.other),
(other.name, other.age, other.other))
This simplicity seems to meet my needs much better than overloading all 6(!) of the rich comparisons. (However, you can get it down to “just” 4 if you rely on the “swapped argument”/reflected behavior, but that results in a net increase of complication, in my humble opinion.)
Are there any unforeseen pitfalls I need to be made aware of if I only overload __cmp__?
I understand the <, <=, ==, etc. operators can be overloaded for other purposes, and can return any object they like. I am not asking about the merits of that approach, but only about differences when using these operators for comparisons in the same sense that they mean for numbers.
Update: As Christopher pointed out, cmp is disappearing in 3.x. Are there any alternatives that make implementing comparisons as easy as the above __cmp__?
Answers:
This is covered by PEP 207 – Rich Comparisons
Also, __cmp__ goes away in python 3.0. ( Note that it is not present on http://docs.python.org/3.0/reference/datamodel.html but it IS on http://docs.python.org/2.7/reference/datamodel.html )
Yep, it’s easy to implement everything in terms of e.g. __lt__ with a mixin class (or a metaclass, or a class decorator if your taste runs that way).
For example:
class ComparableMixin:
def __eq__(self, other):
return not self<other and not other<self
def __ne__(self, other):
return self<other or other<self
def __gt__(self, other):
return other<self
def __ge__(self, other):
return not self<other
def __le__(self, other):
return not other<self
Now your class can define just __lt__ and multiply inherit from ComparableMixin (after whatever other bases it needs, if any). A class decorator would be quite similar, just inserting similar functions as attributes of the new class it’s decorating (the result might be microscopically faster at runtime, at equally minute cost in terms of memory).
Of course, if your class has some particularly fast way to implement (e.g.) __eq__ and __ne__, it should define them directly so the mixin’s versions are not use (for example, that is the case for dict) — in fact __ne__ might well be defined to facilitate that as:
def __ne__(self, other):
return not self == other
but in the code above I wanted to keep the pleasing symmetry of only using <;-).
As to why __cmp__ had to go, since we did have __lt__ and friends, why keep another, different way to do exactly the same thing around? It’s just so much dead-weight in every Python runtime (Classic, Jython, IronPython, PyPy, …). The code that definitely won’t have bugs is the code that isn’t there — whence Python’s principle that there ought to be ideally one obvious way to perform a task (C has the same principle in the “Spirit of C” section of the ISO standard, btw).
This doesn’t mean we go out of our way to prohibit things (e.g., near-equivalence between mixins and class decorators for some uses), but it definitely does mean that we don’t like to carry around code in the compilers and/or runtimes that redundantly exists just to support multiple equivalent approaches to perform exactly the same task.
Further edit: there’s actually an even better way to provide comparison AND hashing for many classes, including that in the question — a __key__ method, as I mentioned on my comment to the question. Since I never got around to writing the PEP for it, you must currently implement it with a Mixin (&c) if you like it:
class KeyedMixin:
def __lt__(self, other):
return self.__key__() < other.__key__()
# and so on for other comparators, as above, plus:
def __hash__(self):
return hash(self.__key__())
It’s a very common case for an instance’s comparisons with other instances to boil down to comparing a tuple for each with a few fields — and then, hashing should be implemented on exactly the same basis. The __key__ special method addresses that need directly.
(Edited 6/17/17 to take comments into account.)
I tried out the comparable mixin answer above. I ran into trouble with “None”. Here is a modified version that handles equality comparisons with “None”. (I saw no reason to bother with inequality comparisons with None as lacking semantics):
class ComparableMixin(object):
def __eq__(self, other):
if not isinstance(other, type(self)):
return NotImplemented
else:
return not self<other and not other<self
def __ne__(self, other):
return not __eq__(self, other)
def __gt__(self, other):
if not isinstance(other, type(self)):
return NotImplemented
else:
return other<self
def __ge__(self, other):
if not isinstance(other, type(self)):
return NotImplemented
else:
return not self<other
def __le__(self, other):
if not isinstance(other, type(self)):
return NotImplemented
else:
return not other<self
To simplify this case there’s a class decorator in Python 2.7+/3.2+, functools.total_ordering, that can be used to implement what Alex suggests. Example from the docs:
@total_ordering
class Student:
def __eq__(self, other):
return ((self.lastname.lower(), self.firstname.lower()) ==
(other.lastname.lower(), other.firstname.lower()))
def __lt__(self, other):
return ((self.lastname.lower(), self.firstname.lower()) <
(other.lastname.lower(), other.firstname.lower()))
Inspired by Alex Martelli’s ComparableMixin & KeyedMixin answers, I came up with the following mixin.
It allows you to implement a single _compare_to() method, which uses key-based comparisons
similar to KeyedMixin, but allows your class to pick the most efficient comparison key based on the type of other. (Note that this mixin doesn’t help much for objects which can be tested for equality but not order).
class ComparableMixin(object):
"""mixin which implements rich comparison operators in terms of a single _compare_to() helper"""
def _compare_to(self, other):
"""return keys to compare self to other.
if self and other are comparable, this function
should return ``(self key, other key)``.
if they aren't, it should return ``None`` instead.
"""
raise NotImplementedError("_compare_to() must be implemented by subclass")
def __eq__(self, other):
keys = self._compare_to(other)
return keys[0] == keys[1] if keys else NotImplemented
def __ne__(self, other):
return not self == other
def __lt__(self, other):
keys = self._compare_to(other)
return keys[0] < keys[1] if keys else NotImplemented
def __le__(self, other):
keys = self._compare_to(other)
return keys[0] <= keys[1] if keys else NotImplemented
def __gt__(self, other):
keys = self._compare_to(other)
return keys[0] > keys[1] if keys else NotImplemented
def __ge__(self, other):
keys = self._compare_to(other)
return keys[0] >= keys[1] if keys else NotImplemented
Python 2.x has two ways to overload comparison operators, __cmp__ or the “rich comparison operators” such as __lt__. The rich comparison overloads are said to be preferred, but why is this so?
Rich comparison operators are simpler to implement each, but you must implement several of them with nearly identical logic. However, if you can use the builtin cmp and tuple ordering, then __cmp__ gets quite simple and fulfills all the comparisons:
class A(object):
def __init__(self, name, age, other):
self.name = name
self.age = age
self.other = other
def __cmp__(self, other):
assert isinstance(other, A) # assumption for this example
return cmp((self.name, self.age, self.other),
(other.name, other.age, other.other))
This simplicity seems to meet my needs much better than overloading all 6(!) of the rich comparisons. (However, you can get it down to “just” 4 if you rely on the “swapped argument”/reflected behavior, but that results in a net increase of complication, in my humble opinion.)
Are there any unforeseen pitfalls I need to be made aware of if I only overload __cmp__?
I understand the <, <=, ==, etc. operators can be overloaded for other purposes, and can return any object they like. I am not asking about the merits of that approach, but only about differences when using these operators for comparisons in the same sense that they mean for numbers.
Update: As Christopher pointed out, cmp is disappearing in 3.x. Are there any alternatives that make implementing comparisons as easy as the above __cmp__?
This is covered by PEP 207 – Rich Comparisons
Also, __cmp__ goes away in python 3.0. ( Note that it is not present on http://docs.python.org/3.0/reference/datamodel.html but it IS on http://docs.python.org/2.7/reference/datamodel.html )
Yep, it’s easy to implement everything in terms of e.g. __lt__ with a mixin class (or a metaclass, or a class decorator if your taste runs that way).
For example:
class ComparableMixin:
def __eq__(self, other):
return not self<other and not other<self
def __ne__(self, other):
return self<other or other<self
def __gt__(self, other):
return other<self
def __ge__(self, other):
return not self<other
def __le__(self, other):
return not other<self
Now your class can define just __lt__ and multiply inherit from ComparableMixin (after whatever other bases it needs, if any). A class decorator would be quite similar, just inserting similar functions as attributes of the new class it’s decorating (the result might be microscopically faster at runtime, at equally minute cost in terms of memory).
Of course, if your class has some particularly fast way to implement (e.g.) __eq__ and __ne__, it should define them directly so the mixin’s versions are not use (for example, that is the case for dict) — in fact __ne__ might well be defined to facilitate that as:
def __ne__(self, other):
return not self == other
but in the code above I wanted to keep the pleasing symmetry of only using <;-).
As to why __cmp__ had to go, since we did have __lt__ and friends, why keep another, different way to do exactly the same thing around? It’s just so much dead-weight in every Python runtime (Classic, Jython, IronPython, PyPy, …). The code that definitely won’t have bugs is the code that isn’t there — whence Python’s principle that there ought to be ideally one obvious way to perform a task (C has the same principle in the “Spirit of C” section of the ISO standard, btw).
This doesn’t mean we go out of our way to prohibit things (e.g., near-equivalence between mixins and class decorators for some uses), but it definitely does mean that we don’t like to carry around code in the compilers and/or runtimes that redundantly exists just to support multiple equivalent approaches to perform exactly the same task.
Further edit: there’s actually an even better way to provide comparison AND hashing for many classes, including that in the question — a __key__ method, as I mentioned on my comment to the question. Since I never got around to writing the PEP for it, you must currently implement it with a Mixin (&c) if you like it:
class KeyedMixin:
def __lt__(self, other):
return self.__key__() < other.__key__()
# and so on for other comparators, as above, plus:
def __hash__(self):
return hash(self.__key__())
It’s a very common case for an instance’s comparisons with other instances to boil down to comparing a tuple for each with a few fields — and then, hashing should be implemented on exactly the same basis. The __key__ special method addresses that need directly.
(Edited 6/17/17 to take comments into account.)
I tried out the comparable mixin answer above. I ran into trouble with “None”. Here is a modified version that handles equality comparisons with “None”. (I saw no reason to bother with inequality comparisons with None as lacking semantics):
class ComparableMixin(object):
def __eq__(self, other):
if not isinstance(other, type(self)):
return NotImplemented
else:
return not self<other and not other<self
def __ne__(self, other):
return not __eq__(self, other)
def __gt__(self, other):
if not isinstance(other, type(self)):
return NotImplemented
else:
return other<self
def __ge__(self, other):
if not isinstance(other, type(self)):
return NotImplemented
else:
return not self<other
def __le__(self, other):
if not isinstance(other, type(self)):
return NotImplemented
else:
return not other<self
To simplify this case there’s a class decorator in Python 2.7+/3.2+, functools.total_ordering, that can be used to implement what Alex suggests. Example from the docs:
@total_ordering
class Student:
def __eq__(self, other):
return ((self.lastname.lower(), self.firstname.lower()) ==
(other.lastname.lower(), other.firstname.lower()))
def __lt__(self, other):
return ((self.lastname.lower(), self.firstname.lower()) <
(other.lastname.lower(), other.firstname.lower()))
Inspired by Alex Martelli’s ComparableMixin & KeyedMixin answers, I came up with the following mixin.
It allows you to implement a single _compare_to() method, which uses key-based comparisons
similar to KeyedMixin, but allows your class to pick the most efficient comparison key based on the type of other. (Note that this mixin doesn’t help much for objects which can be tested for equality but not order).
class ComparableMixin(object):
"""mixin which implements rich comparison operators in terms of a single _compare_to() helper"""
def _compare_to(self, other):
"""return keys to compare self to other.
if self and other are comparable, this function
should return ``(self key, other key)``.
if they aren't, it should return ``None`` instead.
"""
raise NotImplementedError("_compare_to() must be implemented by subclass")
def __eq__(self, other):
keys = self._compare_to(other)
return keys[0] == keys[1] if keys else NotImplemented
def __ne__(self, other):
return not self == other
def __lt__(self, other):
keys = self._compare_to(other)
return keys[0] < keys[1] if keys else NotImplemented
def __le__(self, other):
keys = self._compare_to(other)
return keys[0] <= keys[1] if keys else NotImplemented
def __gt__(self, other):
keys = self._compare_to(other)
return keys[0] > keys[1] if keys else NotImplemented
def __ge__(self, other):
keys = self._compare_to(other)
return keys[0] >= keys[1] if keys else NotImplemented