How does Python's "super" do the right thing?

Question

I’m running Python 2.5, so this question may not apply to Python 3. When you make a diamond class hierarchy using multiple inheritance and create an object of the derived-most class, Python does the Right Thing (TM). It calls the constructor for the derived-most class, then its parent classes as listed from left to right, then the grandparent. I’m familiar with Python’s MRO; that’s not my question. I’m curious how the object returned from super actually manages to communicate to calls of super in the parent classes the correct order. Consider this example code:

#!/usr/bin/python

class A(object):
    def __init__(self): print "A init"

class B(A):
    def __init__(self):
        print "B init"
        super(B, self).__init__()

class C(A):
    def __init__(self):
        print "C init"
        super(C, self).__init__()

class D(B, C):
    def __init__(self):
        print "D init"
        super(D, self).__init__()

x = D()

The code does the intuitive thing, it prints:

D init
B init
C init
A init

However, if you comment out the call to super in B’s init function, neither A nor C’s init function is called. This means B’s call to super is somehow aware of C’s existence in the overall class hierarchy. I know that super returns a proxy object with an overloaded get operator, but how does the object returned by super in D’s init definition communicate the existence of C to the object returned by super in B’s init definition? Is the information that subsequent calls of super use stored on the object itself? If so, why isn’t super instead self.super?

Edit: Jekke quite rightly pointed out that it’s not self.super because super is an attribute of the class, not an instance of the class. Conceptually this makes sense, but in practice super isn’t an attribute of the class either! You can test this in the interpreter by making two classes A and B, where B inherits from A, and calling dir(B). It has no super or __super__ attributes.

Asked By: Joseph Garvin

||

Source

Answer 1

just guessing:

self in all the four methods refer to the same object, that is, of class D.
so, in B.__init__(), the call to to super(B,self) knows the whole diamond ancestry of self and it has to fetch the method from ‘after’ B. in this case, it’s the C class.

Answered By: Javier

Answer 2

Change your code to this and I think it’ll explain things (presumably super is looking at where, say, B is in the __mro__?):

class A(object):
    def __init__(self):
        print "A init"
        print self.__class__.__mro__

class B(A):
    def __init__(self):
        print "B init"
        print self.__class__.__mro__
        super(B, self).__init__()

class C(A):
    def __init__(self):
        print "C init"
        print self.__class__.__mro__
        super(C, self).__init__()

class D(B, C):
    def __init__(self):
        print "D init"
        print self.__class__.__mro__
        super(D, self).__init__()

x = D()

If you run it you’ll see:

D init
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <type 'object'>)
B init
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <type 'object'>)
C init
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <type 'object'>)
A init
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <type 'object'>)

Also it’s worth checking out Python’s Super is nifty, but you can’t use it.

Answered By: Jacob Gabrielson

Answer 3

I have provided a bunch of links below, that answer your question in more detail and more precisely than I can ever hope to. I will however give an answer to your question in my own words as well, to save you some time. I’ll put it in points –

super is a builtin function, not an attribute.
Every type (class) in Python has an __mro__ attribute, that stores the method resolution order of that particular instance.
Each call to super is of the form super(type[, object-or-type]). Let us assume that the second attribute is an object for the moment.
At the starting point of super calls, the object is of the type of the Derived class (say DC).
super looks for methods that match (in your case __init__) in the classes in the MRO, after the class specified as the first argument (in this case classes after DC).
When the matching method is found (say in class BC1), it is called.
(This method should use super, so I am assuming it does – See Python’s super is nifty but can’t be used – link below)
That method then causes a search in the object’s class’ MRO for the next method, to the right of BC1.
Rinse wash repeat till all methods are found and called.

Explanation for your example

 MRO: D,B,C,A,object

super(D, self).__init__() is called. isinstance(self, D) => True
Search for next method in the MRO in classes to the right of D.

B.__init__ found and called

B.__init__ calls super(B, self).__init__().

isinstance(self, B) => False
isinstance(self, D) => True
Thus, the MRO is the same, but the search continues to the right of B i.e. C,A,object are searched one by one. The next __init__ found is called.
And so on and so forth.

An explanation of super
http://www.python.org/download/releases/2.2.3/descrintro/#cooperation
Things to watch for when using super
http://fuhm.net/super-harmful/
Pythons MRO Algorithm:
http://www.python.org/download/releases/2.3/mro/
super’s docs:
http://docs.python.org/library/functions.html
The bottom of this page has a nice section on super:
http://docstore.mik.ua/orelly/other/python/0596001886_pythonian-chp-5-sect-2.html

I hope this helps clear it up.

Answered By: batbrat

Answer 4

super() knows the full class hierarchy. This is what happens inside B’s init:

>>> super(B, self)
<super: <class 'B'>, <D object>>

This resolves the central question,

how does the object returned by super in D’s init definition communicate the existence of C to the object returned by super in B’s init definition?

Namely, in B’s init definition, self is an instance of D, and thus communicates the existence of C. For example C can be found in type(self).__mro__.

Answered By: hrr

Answer 5

Jacob’s answer shows how to understand the problem, while batbrat’s shows the details and hrr’s goes straight to the point.

One thing they do not cover (at least not explicity) from your question is this point:

However, if you comment out the call to super in B’s init function, neither A nor C’s init function is called.

To understand that, change Jacob’s code to to print the stack on A’s init, as below:

import traceback

class A(object):
    def __init__(self):
        print "A init"
        print self.__class__.__mro__
        traceback.print_stack()

class B(A):
    def __init__(self):
        print "B init"
        print self.__class__.__mro__
        super(B, self).__init__()

class C(A):
    def __init__(self):
        print "C init"
        print self.__class__.__mro__
        super(C, self).__init__()

class D(B, C):
    def __init__(self):
        print "D init"
        print self.__class__.__mro__
        super(D, self).__init__()

x = D()

It is a bit surprising to see that B‘s line super(B, self).__init__() is actually calling C.__init__(), as C is not a baseclass of B.

D init
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <type 'object'>)
B init
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <type 'object'>)
C init
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <type 'object'>)
A init
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <type 'object'>)
  File "/tmp/jacobs.py", line 31, in <module>
    x = D()
  File "/tmp/jacobs.py", line 29, in __init__
    super(D, self).__init__()
  File "/tmp/jacobs.py", line 17, in __init__
    super(B, self).__init__()
  File "/tmp/jacobs.py", line 23, in __init__
    super(C, self).__init__()
  File "/tmp/jacobs.py", line 11, in __init__
    traceback.print_stack()

This happens because super (B, self) is not ‘calling the B’s baseclass version of __init__‘. Instead, it is ‘calling __init__ on the first class to the right of B that is present on self‘s __mro__ and that has such an attribute.

So, if you comment out the call to super in B’s init function, the method stack will stop on B.__init__, and will never reach C or A.

To summarize:

Regardless of which class is referring to it, self is always a reference to the instance, and its __mro__ and __class__ remain constant
super() finds the method looking to the classes that are to the right of the current one on the __mro__. As the __mro__ remains constant, what happens is that it is searched as a list, not as a tree or a graph.

On that last point, note that the full name of the MRO’s algorithm is C3 superclass linearization. That is, it flattens that structure into a list. When the different super() calls happen, they are effectivelly iterating that list.

Answered By: caxcaxcoatl

How does Python's "super" do the right thing?

Question:

Answers: