Using a class' __new__ method as a Factory: __init__ gets called twice

Question:

I encountered a strange bug in python where using the __new__ method of a class as a factory would lead to the __init__ method of the instantiated class to be called twice.

The idea was originally to use the __new__ method of the mother class to return a specific instance of one of her children depending on the parameters that are passed, without having to declare a factory function outside of the class.

I know that using a factory function would be the best design-pattern to use here, but changing the design pattern at this point of the project would be costly. My question hence is: is there a way to avoid the double call to __init__ and get only a single call to __init__ in such a schema ?

class Shape(object):
    def __new__(cls, desc):
        if cls is Shape:
            if desc == 'big':   return Rectangle(desc)
            if desc == 'small': return Triangle(desc)
        else:
            return super(Shape, cls).__new__(cls, desc)

    def __init__(self, desc):
        print "init called"
        self.desc = desc

class Triangle(Shape):
    @property
    def number_of_edges(self): return 3

class Rectangle(Shape):
    @property
    def number_of_edges(self): return 4

instance = Shape('small')
print instance.number_of_edges

>>> init called
>>> init called
>>> 3

Any help greatly appreciated.

Asked By: xApple

||
Source

Answers:

I can’t actually reproduce this behavior in either of the Python interpreters I have installed, so this is something of a guess. However…

__init__ is being called twice because you are initializing two objects: the original Shape object, and then one of its subclasess. If you change your __init__ so it also prints the class of the object being initialized, you will see this.

print type(self), "init called"

This is harmless because the original Shape will be discarded, since you are not returning a reference to it in your __new__().

Since calling a function is syntactically identical to instantiating a class, you can change this to a function without changing anything else, and I recommend that you do exactly that. I don’t understand your reluctance.

Answered By: kindall

When you construct an object Python calls its __new__ method to create the object then calls __init__ on the object that is returned. When you create the object from inside __new__ by calling Triangle() that will result in further calls to __new__ and __init__.

What you should do is:

class Shape(object):
    def __new__(cls, desc):
        if cls is Shape:
            if desc == 'big':   return super(Shape, cls).__new__(Rectangle)
            if desc == 'small': return super(Shape, cls).__new__(Triangle)
        else:
            return super(Shape, cls).__new__(cls, desc)

which will create a Rectangle or Triangle without triggering a call to __init__ and then __init__ is called only once.

Edit to answer @Adrian’s question about how super works:

super(Shape,cls) searches cls.__mro__ to find Shape and then searches down the remainder of the sequence to find the attribute.

Triangle.__mro__ is (Triangle, Shape, object) and
Rectangle.__mro__ is (Rectangle, Shape, object) while Shape.__mro__ is just (Shape, object).
For any of those cases when you call super(Shape, cls) it ignores everything in the mro squence up to and including Shape so the only thing left is the single element tuple (object,) and that is used to find the desired attribute.

This would get more complicated if you had a diamond inheritance:

class A(object): pass
class B(A): pass
class C(A): pass
class D(B,C): pass

now a method in B might use super(B, cls) and if it were a B instance would search (A, object) but if you had a D instance the same call in B would search (C, A, object) because the D.__mro__ is (B, C, A, object).

So in this particular case you could define a new mixin class that modifies the construction behaviour of the shapes and you could have specialised triangles and rectangles inheriting from the existing ones but constructed differently.

Answered By: Duncan

After posting my question, I continued searching for a solution an found a way to solve the problem that looks like a bit of a hack. It is inferior to Duncan’s solution, but I thought it could be interesting to mention none the less. The Shapeclass becomes:

class ShapeFactory(type):
    def __call__(cls, desc):
        if cls is Shape:
            if desc == 'big':   return Rectangle(desc)
            if desc == 'small': return Triangle(desc)
        return type.__call__(cls, desc)

class Shape(object):
    __metaclass__ = ShapeFactory 
    def __init__(self, desc):
        print "init called"
        self.desc = desc
Answered By: xApple