Is super() broken in Python-2.x?
Question:
It’s often stated that super should be avoided in Python 2. I’ve found in my use of super in Python 2 that it never acts the way I expect unless I provide all arguments such as the example:
super(ThisClass, self).some_func(*args, **kwargs)
It seems to me this defeats the purpose of using super(), it’s neither more concise, or much better than TheBaseClass.some_func(self, *args, **kwargs). For most purposes method resolution order is a distant fairy tale.
- Other than the fact that 2.7 is the last major release to Python 2, why does super remain broken in Python 2?
- How and why has Python 3’s super changed? Are there any caveats?
- When and why should I use
super going forward?
Answers:
You seem to imply in your post that
def some_func(self, *args, **kwargs):
self.__class__.some_func(self, *args, **kwargs)
is not an infinite recursion. It is, and super would be more correct.
Also, yes, you are required to pass all arguments to super(). This is a bit like complaining that max() doesn’t work like expected unless you pass it all the numbers you want to check.
In 3.x, however, fewer arguments are needed: you can do super().foo(*args, **kwargs) instead of super(ThisClass, self).foo(*args, **kwargs).
Anyway, I’m unsure as to any situations when super should be avoided. Its behavior is only “weird” when MI is involved, and when MI is involved, super() is basically your only hope for a correct solution. In Single-Inheritance it’s just slightly wordier than SuperClass.foo(self, *args, **kwargs), and does nothing different.
I think I agree with Sven that this sort of MI is worth avoiding, but I don’t agree that super is worth avoiding. If your class is supposed to be inherited, super offers users of your class hope of getting MI to work, if they’re weird in that way, so it makes your class more usable.
super() is not broken — it just should not be considered the standard way of calling a method of the base class. This did not change with Python 3.x. The only thing that changed is that you don’t need to pass the arguments self, cls in the standard case that self is the first parameter of the current function and cls is the class currently being defined.
Regarding your question when to actually use super(), my answer would be: hardly ever. I personally try to avoid the kind of multiple inheritance that would make super() useful.
Edit: An example from real life that I once ran into: I had some classes defining a run() method, some of which had base classes. I used super() to call the inherited constructors — I did not think it mattered because I was using single inheritance only:
class A(object):
def __init__(self, i):
self.i = i
def run(self, value):
return self.i * value
class B(A):
def __init__(self, i, j):
super(B, self).__init__(i)
self.j = j
def run(self, value):
return super(B, self).run(value) + self.j
Just imagine there were several of these classes, all with individual constructor prototypes, and all with the same interface to run().
Now I wanted to add some additional functionality to all of these classes, say logging. The additional functionality required an additional method to be defined on all these classes, say info(). I did not want to invade the original classes, but rather define a second set of classes inheriting from the original ones, adding the info() method and inheriting from a mix-in providing the actual logging. Now, I could not use super() in the constructor any more, so I used direct calls:
class Logger(object):
def __init__(self, name):
self.name = name
def run_logged(self, value):
print "Running", self.name, "with info", self.info()
return self.run(value)
class BLogged(B, Logger):
def __init__(self, i, j):
B.__init__(self, i, j)
Logger.__init__("B")
def info(self):
return 42
Here things stop working. The super() call in the base class constructor suddenly calls Logger.__init__(), and BLogged can’t do anything about it. There is actually no way to make this work, except for removing the super() call in B itself.
[Another Edit: I don’t seem to have made my point, judging from all the comments here and below the other answers. Here is how to make this code work using super():
class A(object):
def __init__(self, i, **kwargs):
super(A, self).__init__(**kwargs)
self.i = i
def run(self, value):
return self.i * value
class B(A):
def __init__(self, j, **kwargs):
super(B, self).__init__(**kwargs)
self.j = j
def run(self, value):
return super(B, self).run(value) + self.j
class Logger(object):
def __init__(self, name, **kwargs):
super(Logger,self).__init__(**kwargs)
self.name = name
def run_logged(self, value):
print "Running", self.name, "with info", self.info()
return self.run(value)
class BLogged(B, Logger):
def __init__(self, **kwargs):
super(BLogged, self).__init__(name="B", **kwargs)
def info(self):
return 42
b = BLogged(i=3, j=4)
Compare this with the use of explicit superclass calls. You decide which version you prefer.]
This and similar stories are why I think that super() should not be considered the standard way of calling methods of the base class. It does not mean super() is broken.
Did you read the article that you link it? It doesn’t conclude that super should be avoided but that you should be wary of its caveats when using it. These caveats are summarized by the article, though I would disagree with their suggestions.
The main point of the article is that multiple inheritance can get messy, and super doesn’t help as much as the author would want. However doing multiple inheritance without super is often even more complicated.
If you’re not doing multiple inheritance, super gives you the advantage that anyone inheriting from your class can add simple mixins and their __init__ would be properly called. Just remember to always call the __init__ of the superclass, even when you’re inheriting from object, and to pass all the remaining arguments (*a and **kw) to it. When you’re calling other methods from the parent class also use super, but this time use their proper signature that you already know (i.e. ensure that they have the same signature in all classes).
If you’re doing multiple inheritance you’d have to dig deeper than that, and probably re-read the same article more carefully to be aware of the caveats. And it’s also only during multiple inheritance when you might a situation where an explicit call to the parent might be better than super, but without a specific scenario nobody can tell you whether super should be used or not.
The only change in super in Python 3.x is that you don’t need to explicitly pass the current class and self to it. This makes super more attractive, because using it would mean no hardcoding of either the parent class or the current class.
super() is not broken, in Python 2 or Python 3.
Let’s consider the arguments from the blog post:
- It doesn’t do what it sounds like it does.
OK, you may agree or disagree on that, it’s pretty subjective. What should it have been called then? super() is a replacement for calling the superclass directly, so the name seems fine to me. It does NOT call the superclass directly, because if that was all it did, it would be pointless, as you could do that anyway. OK, admittedly, that may not be obvious, but the cases where you need super() are generally not obvious. If you need it, you are doing some pretty hairy multiple inheritance. It’s not going to be obvious. (Or you are doing a simple mixin, in which case it will be pretty obvious and behave as you expect even if you didn’t read the docs).
If you can call the superclass directly, that’s probably what you’ll end up doing. That’s the easy and intuitive way of doing it. super() only comes into play when that doesn’t work.
- It doesn’t mesh well with calling the superclass directly.
Yes, because it’s designed to solve a problem with doing that. You can call the superclass directly if, and only if, you know exactly what class that is. Which you don’t for mixins, for example, or when your class hierarchy is so messed up that you actually are merging two branches (which is the typical example in all examples of using super()).
So as long as every class in your class hierarchy has a well defined place, calling the superclass directly works. If you don’t, then it does not work, and in that case you must use super() instead. That’s the point of super() that it figures out what the “next superclass” is according to the MRO, without you explicitly having to specify it, because you can’t always do that because you don’t always know what it is, for example when using mixins.
- The completely different programming language Dylan, a sort of lisp-thingy, solves this in another way that can’t be used in Python because it’s very different.
Eh. OK?
super() doesn’t call your superclass.
Yeah, you said that.
- Don’t mix
super() and direct calling.
Yeah, you said that too.
So, there is two arguments against it: 1. The name is bad. 2. You have to use it consistently.
That does not translate to it being “broken” or that it should be “avoided”.
@Sven Marnach:
The problem with your example is that you mix explicit superclass calls B.__init__ and Logger.__init__ in Blogged with super() in B. That won’t work. Either you use all explicit superclass calls or use super() on all classes. When you use super() you need to use it on all classes involved, including A I think. Also in your example I think you could use explicit superclass calls in all classes, i.e use A.__init__ in class B.
When there is no diamond inheritance I think super() doesn’t have much advantage. The problem is, however, that you don’t know in advance if you will get into any diamond inheritance in the future so in that case it would be wise to use super() anyway (but then use it consistently). Otherwise you would end up having to change all classes at a later time or run into problems.
It’s often stated that super should be avoided in Python 2. I’ve found in my use of super in Python 2 that it never acts the way I expect unless I provide all arguments such as the example:
super(ThisClass, self).some_func(*args, **kwargs)
It seems to me this defeats the purpose of using super(), it’s neither more concise, or much better than TheBaseClass.some_func(self, *args, **kwargs). For most purposes method resolution order is a distant fairy tale.
- Other than the fact that 2.7 is the last major release to Python 2, why does super remain broken in Python 2?
- How and why has Python 3’s super changed? Are there any caveats?
- When and why should I use
supergoing forward?
You seem to imply in your post that
def some_func(self, *args, **kwargs):
self.__class__.some_func(self, *args, **kwargs)
is not an infinite recursion. It is, and super would be more correct.
Also, yes, you are required to pass all arguments to super(). This is a bit like complaining that max() doesn’t work like expected unless you pass it all the numbers you want to check.
In 3.x, however, fewer arguments are needed: you can do super().foo(*args, **kwargs) instead of super(ThisClass, self).foo(*args, **kwargs).
Anyway, I’m unsure as to any situations when super should be avoided. Its behavior is only “weird” when MI is involved, and when MI is involved, super() is basically your only hope for a correct solution. In Single-Inheritance it’s just slightly wordier than SuperClass.foo(self, *args, **kwargs), and does nothing different.
I think I agree with Sven that this sort of MI is worth avoiding, but I don’t agree that super is worth avoiding. If your class is supposed to be inherited, super offers users of your class hope of getting MI to work, if they’re weird in that way, so it makes your class more usable.
super() is not broken — it just should not be considered the standard way of calling a method of the base class. This did not change with Python 3.x. The only thing that changed is that you don’t need to pass the arguments self, cls in the standard case that self is the first parameter of the current function and cls is the class currently being defined.
Regarding your question when to actually use super(), my answer would be: hardly ever. I personally try to avoid the kind of multiple inheritance that would make super() useful.
Edit: An example from real life that I once ran into: I had some classes defining a run() method, some of which had base classes. I used super() to call the inherited constructors — I did not think it mattered because I was using single inheritance only:
class A(object):
def __init__(self, i):
self.i = i
def run(self, value):
return self.i * value
class B(A):
def __init__(self, i, j):
super(B, self).__init__(i)
self.j = j
def run(self, value):
return super(B, self).run(value) + self.j
Just imagine there were several of these classes, all with individual constructor prototypes, and all with the same interface to run().
Now I wanted to add some additional functionality to all of these classes, say logging. The additional functionality required an additional method to be defined on all these classes, say info(). I did not want to invade the original classes, but rather define a second set of classes inheriting from the original ones, adding the info() method and inheriting from a mix-in providing the actual logging. Now, I could not use super() in the constructor any more, so I used direct calls:
class Logger(object):
def __init__(self, name):
self.name = name
def run_logged(self, value):
print "Running", self.name, "with info", self.info()
return self.run(value)
class BLogged(B, Logger):
def __init__(self, i, j):
B.__init__(self, i, j)
Logger.__init__("B")
def info(self):
return 42
Here things stop working. The super() call in the base class constructor suddenly calls Logger.__init__(), and BLogged can’t do anything about it. There is actually no way to make this work, except for removing the super() call in B itself.
[Another Edit: I don’t seem to have made my point, judging from all the comments here and below the other answers. Here is how to make this code work using super():
class A(object):
def __init__(self, i, **kwargs):
super(A, self).__init__(**kwargs)
self.i = i
def run(self, value):
return self.i * value
class B(A):
def __init__(self, j, **kwargs):
super(B, self).__init__(**kwargs)
self.j = j
def run(self, value):
return super(B, self).run(value) + self.j
class Logger(object):
def __init__(self, name, **kwargs):
super(Logger,self).__init__(**kwargs)
self.name = name
def run_logged(self, value):
print "Running", self.name, "with info", self.info()
return self.run(value)
class BLogged(B, Logger):
def __init__(self, **kwargs):
super(BLogged, self).__init__(name="B", **kwargs)
def info(self):
return 42
b = BLogged(i=3, j=4)
Compare this with the use of explicit superclass calls. You decide which version you prefer.]
This and similar stories are why I think that super() should not be considered the standard way of calling methods of the base class. It does not mean super() is broken.
Did you read the article that you link it? It doesn’t conclude that super should be avoided but that you should be wary of its caveats when using it. These caveats are summarized by the article, though I would disagree with their suggestions.
The main point of the article is that multiple inheritance can get messy, and super doesn’t help as much as the author would want. However doing multiple inheritance without super is often even more complicated.
If you’re not doing multiple inheritance, super gives you the advantage that anyone inheriting from your class can add simple mixins and their __init__ would be properly called. Just remember to always call the __init__ of the superclass, even when you’re inheriting from object, and to pass all the remaining arguments (*a and **kw) to it. When you’re calling other methods from the parent class also use super, but this time use their proper signature that you already know (i.e. ensure that they have the same signature in all classes).
If you’re doing multiple inheritance you’d have to dig deeper than that, and probably re-read the same article more carefully to be aware of the caveats. And it’s also only during multiple inheritance when you might a situation where an explicit call to the parent might be better than super, but without a specific scenario nobody can tell you whether super should be used or not.
The only change in super in Python 3.x is that you don’t need to explicitly pass the current class and self to it. This makes super more attractive, because using it would mean no hardcoding of either the parent class or the current class.
super() is not broken, in Python 2 or Python 3.
Let’s consider the arguments from the blog post:
- It doesn’t do what it sounds like it does.
OK, you may agree or disagree on that, it’s pretty subjective. What should it have been called then? super() is a replacement for calling the superclass directly, so the name seems fine to me. It does NOT call the superclass directly, because if that was all it did, it would be pointless, as you could do that anyway. OK, admittedly, that may not be obvious, but the cases where you need super() are generally not obvious. If you need it, you are doing some pretty hairy multiple inheritance. It’s not going to be obvious. (Or you are doing a simple mixin, in which case it will be pretty obvious and behave as you expect even if you didn’t read the docs).
If you can call the superclass directly, that’s probably what you’ll end up doing. That’s the easy and intuitive way of doing it. super() only comes into play when that doesn’t work.
- It doesn’t mesh well with calling the superclass directly.
Yes, because it’s designed to solve a problem with doing that. You can call the superclass directly if, and only if, you know exactly what class that is. Which you don’t for mixins, for example, or when your class hierarchy is so messed up that you actually are merging two branches (which is the typical example in all examples of using super()).
So as long as every class in your class hierarchy has a well defined place, calling the superclass directly works. If you don’t, then it does not work, and in that case you must use super() instead. That’s the point of super() that it figures out what the “next superclass” is according to the MRO, without you explicitly having to specify it, because you can’t always do that because you don’t always know what it is, for example when using mixins.
- The completely different programming language Dylan, a sort of lisp-thingy, solves this in another way that can’t be used in Python because it’s very different.
Eh. OK?
super()doesn’t call your superclass.
Yeah, you said that.
- Don’t mix
super()and direct calling.
Yeah, you said that too.
So, there is two arguments against it: 1. The name is bad. 2. You have to use it consistently.
That does not translate to it being “broken” or that it should be “avoided”.
@Sven Marnach:
The problem with your example is that you mix explicit superclass calls B.__init__ and Logger.__init__ in Blogged with super() in B. That won’t work. Either you use all explicit superclass calls or use super() on all classes. When you use super() you need to use it on all classes involved, including A I think. Also in your example I think you could use explicit superclass calls in all classes, i.e use A.__init__ in class B.
When there is no diamond inheritance I think super() doesn’t have much advantage. The problem is, however, that you don’t know in advance if you will get into any diamond inheritance in the future so in that case it would be wise to use super() anyway (but then use it consistently). Otherwise you would end up having to change all classes at a later time or run into problems.