What's “`__new__“` by default in Python 3?

Question:

I believe I have some sort of understanding of what __new__ is supposed to do (create an instance, of a class, but not initialize it, that is the job of __init__). I’d like to understand, however, what Python 3 implements as a __new__ method by default.

I also find it somewhat confusing that cls is an argument for __new__, but __new__ is a staticmethod and not a classmethod (I got this from the documentation). How does it know that it is being passed a type as its first argument?

Asked By: Ignacio

||
Source

Answers:

Based on my understanding the default implementation of __new__() is something like this

class Default(object):
def __new__(cls, *args, **kwargs):
    print("In new")
    return super().__new__(cls,*args, **kwargs)

def __init__(self):
    print("In init default")

default = Default()
print(type(default))

Output

In new
In init default
<class '__main__.Default'>

Based on the documentation
https://docs.python.org/3/reference/datamodel.html#object.new

Typical implementations create a new instance of the class by invoking the superclass’s __new__() method using super().new(cls[, …]) with appropriate arguments and then modifying the newly-created instance as necessary before returning it.

The super().__new__() call will create the instance and __init__() will initialize.

Below code shows an incorrect overriding of __new__()

class InccorectOverride(object):

def __new__(cls, *args, **kwargs):
    pass

def __init__(self):
    print("In init InccorectOverride")

a = InccorectOverride()
print(type(a))

Output

<class 'NoneType'>

Since __new__() does not return the instance, the output has a value of NoneType

Hope this answers your question

Answered By: Shishir

First part: what does __new__ do by default? Because the creation of an object from scratch is a fundamental, implementation-specific operation, the definition of object.__new__ is necessarily (like the rest of the definition of object) part of the implementation itself. That means you need to look in the source code of CPython, PyPy, Cython, etc. to see exactly how object creation is managed in any particular implementation of Python. Typically, it’s all low-level bookkeeping that can’t be accessed directly from Python itself.

Second part: how does __new__ know that it gets a class argument? Because it assumes its first argument is a class, and the caller had better provide a class if they expect __new__ to work correctly! That said, nobody really ever calls __new__ expclitly, except via super in an override of __new__, and then, you have to make sure to pass cls explicitly yourself:

def __new__(cls, *args, **kwargs):
    rv = super().__new__(cls, *args, **kwargs)  # Not super().__new__(*args, **kwargs)

The rest of the time, you create an instance of a class Foo not by calling Foo.__new__(Foo, ...) directly, but just by calling the type itself: Foo(...). This is managed because the __call__ method of Foo‘s metaclass takes care of calling __new__ for your. For example, you can imagine that type.__call__ is defined roughly as

# A regular instance method of type; we use cls instead of self to
# emphasize that an instance of a metaclass is a class
def __call__(cls, *args, **kwargs):
    rv = cls.__new__(cls, *args, **kwargs)  # Because __new__ is static!
    if isinstance(rv, cls):
        rv.__init__(*args, **kwargs)
    return rv

Note that __init__ is only invoked if __new__ actually returns an instance of the class calling __new__ in the first place.

Answered By: chepner

Your question was

If we do not override it, then what does __new__ do in Python?

Suppose that we have a very simple class named Point. We instantiate the Point class as follows:

# construct and initialize paula the point
paula = Point(10.2, 7.4)

In order to help explain what __new__ does, I will write a classmethod named make_point which has almost the same behavior as calls to the class constructor, such as paula = = Point(10.2, 7.4).

class Point():
    def __new__(cls, *args, **kwargs):
        print("__new__(" + ", ".join(str(x) for x in [cls, *args]) + ")")
        obj = super().__new__(cls)
        return obj

    def __init__(self, y:float, y:float):
        args = (y, y)
        print("__init__(" + ", ".join(str(x) for x in [self, *args]) + ")")
        self._y = y
        self._y = y

    @classmethod
    def make_point(cls, *args):
        new_instance = cls.__new__(cls, *args)
        if isinstance(new_instance, cls):
            cls.__init__(new_instance, *args)
        return new_instance

Now we can instantiate the Point class as follows:

peter = Point.make_point(59.87, 5.91)
Answered By: Samuel Muldoon