__new__ and __init__ in Python
Question:
I am learning Python and so far I can tell the things below about __new__ and __init__:
__new__ is for object creation__init__ is for object initialization__new__ is invoked before __init__ as __new__ returns a new instance and __init__ invoked afterwards to initialize inner state.__new__ is good for immutable object as they cannot be changed once they are assigned. So we can return new instance which has new state.- We can use
__new__ and __init__ for both mutable object as its inner state can be changed.
But I have another questions now.
- When I create a new instance such as
a = MyClass("hello","world"), how these arguments are passed? I mean how I should structure the class using __init__ and __new__ as they are different and both accepts arbitrary arguments besides default first argument.
self keyword is in terms of name can be changed to something else? But I am wondering cls is in terms of name is subject to change to something else as it is just a parameter name?
I made a little experiments as such below:
>>> class MyClass(tuple):
def __new__(tuple):
return [1,2,3]
and I did below:
>>> a = MyClass()
>>> a
[1, 2, 3]
Albeit I said I want to return tuple, this code works fine and returned me [1,2,3]. I knew we were passing the first parameters as the type we wanted to receive once the __new__ function is invoked. We are talking about New function right? I don’t know other languages return type other than bound type?
And I did anther things as well:
>>> issubclass(MyClass,list)
False
>>> issubclass(MyClass,tuple)
True
>>> isinstance(a,MyClass)
False
>>> isinstance(a,tuple)
False
>>> isinstance(a,list)
True
I didn’t do more experiment because the further wasn’t bright and I decided to stop there and decided to ask StackOverflow.
The SO posts I read:
Answers:
how I should structure the class using __init__ and __new__ as they are different and both accepts arbitrary arguments besides default first argument.
Only rarely will you have to worry about __new__. Usually, you’ll just define __init__ and let the default __new__ pass the constructor arguments to it.
self keyword is in terms of name can be changed to something else? But I am wondering cls is in terms of name is subject to change to something else as it is just a parameter name?
Both are just parameter names with no special meaning in the language. But their use is a very strong convention in the Python community; most Pythonistas will never change the names self and cls in these contexts and will be confused when someone else does.
Note that your use of def __new__(tuple) re-binds the name tuple inside the constructor function. When actually implementing __new__, you’ll want to do it as
def __new__(cls, *args, **kwargs):
# do allocation to get an object, say, obj
return obj
Albeit I said I want to return tuple, this code works fine and returned me [1,2,3].
MyClass() will have the value that __new__ returns. There’s no implicit type checking in Python; it’s the responsibility of the programmer to return the correct type (“we’re all consenting adults here”). Being able to return a different type than requested can be useful for implementing factories: you can return a subclass of the type requested.
This also explains the issubclass/isinstance behavior you observe: the subclass relationship follows from your use of class MyClass(tuple), the isinstance reflects that you return the “wrong” type from __new__.
For reference, check out the requirements for __new__ in the Python Language Reference.
Edit: ok, here’s an example of potentially useful use of __new__. The class Eel keeps track of how many eels are alive in the process and refuses to allocate if this exceeds some maximum.
class Eel(object):
MAX_EELS = 20
n_eels = 0
def __new__(cls, *args, **kwargs):
if cls.n_eels == cls.MAX_EELS:
raise HovercraftFull()
obj = super(Eel, cls).__new__(cls)
cls.n_eels += 1
return obj
def __init__(self, voltage):
self.voltage = voltage
def __del__(self):
type(self).n_eels -= 1
def electric(self):
"""Is this an electric eel?"""
return self.voltage > 0
Mind you, there are smarter ways to accomplish this behavior.
Your question was something like,
What is the difference between __new__ and __init__ in Python?
Suppose that we have a very simple class named Rectangle
class Rectangle():
def __init__(self, width:float, height:float):
self._width = width
self._hieght = height
We instantiate the Rectangle class as follows:
rocky_the_rectangle = Rectangle(10.01, 8.91)
In order to help explain what __new__ does, I will write a classmethod named make_rectangle which has almost the same behavior as the class-constructor.
class Rectangle():
def __new__(cls, *args, **kwargs):
print("__new__(" + ", ".join(str(x) for x in [cls, *args]) + ")")
obj = super().__new__(cls)
return obj
def __init__(self, width:float, height:float):
args = (width, height)
print("__init__(" + ", ".join(str(x) for x in [self, *args]) + ")")
self._width = width
self._height = height
@classmethod
def make_rectangle(cls, *args):
new_instance = cls.__new__(cls, *args)
if isinstance(new_instance, cls):
cls.__init__(new_instance, *args)
return new_instance
Now we can instantiate the Rectangle class as follows:
rocky_II = Rectangle.make_rectangle(10.01, 8.91)
The __init__ method is unable to replace the self parameter with something new and different.
For example, people often want to put a wrapper around the self parameter. You might want something like this:
import functools
import sys
class decorator:
def __new__(cls, kallable):
instance = super().__new__(cls)
instance = functools.update_wrapper(instance, kallable)
return instance
def __init__(self, kallable):
self._kallable = kallable
self._file = sys.stdout
def __call__(self, *args, **kwargs):
print("__init__(" + ", ".join(str(x) for x in [self, *args]) + ")", file=self._file)
return self._kallable(*args, **kwargs)
@decorator
def pow(base:float, exp:int):
"""
+------------------------------------------+
| EXAMPLES |
+------------------------------------------+
| BASE | EXPONENT | OUTPUT |
+------+----------+------------------------+
| 2 | 5 | 2^5 | 32 |
| 2.5 | 7 | 2.5^7 | 610.3515625 |
| 10 | 3 | 10^3 | 1000 |
| 0.1 | 5 | 0.1^5 | 0.00001 |
| 7 | 0 | 7^0 | 1 |
+------+----------+----------+-------------+
"""
base = float(base)
# convert `exp` to string to avoid flooring, or truncating, floats
exp = int(str(exp))
if exp > 0:
return base * pow(base, exp-1)
else: # exp == 2
return 1
result1 = pow(2, 5)
result2 = pow(8.1, 0)
print("pow(2, 5) == " , result1)
print("pow(8.1, 0) == ", result2)
print("What is the documentation string? The doc-string is... ", pow.__doc__)
It will not help to use functools.update_wrapper inside of __init__. If you try to write the following…
class f:
def __init__(outter, inner):
# `outter` is usually named `self`, but you are...
# ... allowed to use other names for it.
outter = functools.update_wrapper(outter, inner)
… then outter will get ignored. You cannot replace the parameter named self with a different self
__new__ allows us to replace self with a wrapper around self
class decorator:
def __new__(cls, kallable):
instance = super().__new__(cls)
instance = functools.update_wrapper(instance, kallable)
return instance
Without using functools.update_wrapper inside of __new__ the doc-string inside of the original callable will be wiped-out, ignored, not inherited, and/or shadowed.
seems that nobody had cover how the arguments are handled through the instantiation
lets build a simple class that define new and init
censured = ["Cicciogamer"]
class Foo(object):
def __new__(cls, name):
if name in censured:
print("you shouldn't do this")
return super().__new__(cls)
def __init__(self, var):
self.var = var
when you call a class object to get an instance, python implicity call
Foo.__call__(*args, **kwargs)
so with the class above you get the (maybe) undesired double arguments passing:
foo = Foo("Cicciogamer")
>>> "you shouldn't do this"
foo.var
>>> "Cicciogamer"
To achieve the control on how these parameters behave ASFAIK you must override the call method of the CLASS OBJECT not instance
you can use metaclasses but it might be overkill
class MetaFoo:
def __call__(cls, name, var):
foo_object = cls.__new__(name)
cls.__init__(foo_object, var)
return foo_object
class Foo(metaclass=MetaFoo):
...
foo = Foo("Cicciogamer", 505)
>>> "you shouldn't do this"
foo.var
>>> 505
or i think you can achieve this with simply
class Foo:
...
@classmethod
def new(cls, name, var):
foo_object = cls.__new__(name)
cls.__init__(foo_object, var)
return foo_object
__call__ = new
please let me notice if this could be achieved in a better way, or if i simply screwed up something
I am learning Python and so far I can tell the things below about __new__ and __init__:
__new__is for object creation__init__is for object initialization__new__is invoked before__init__as__new__returns a new instance and__init__invoked afterwards to initialize inner state.__new__is good for immutable object as they cannot be changed once they are assigned. So we can return new instance which has new state.- We can use
__new__and__init__for both mutable object as its inner state can be changed.
But I have another questions now.
- When I create a new instance such as
a = MyClass("hello","world"), how these arguments are passed? I mean how I should structure the class using__init__and__new__as they are different and both accepts arbitrary arguments besides default first argument. selfkeyword is in terms of name can be changed to something else? But I am wonderingclsis in terms of name is subject to change to something else as it is just a parameter name?
I made a little experiments as such below:
>>> class MyClass(tuple):
def __new__(tuple):
return [1,2,3]
and I did below:
>>> a = MyClass()
>>> a
[1, 2, 3]
Albeit I said I want to return tuple, this code works fine and returned me [1,2,3]. I knew we were passing the first parameters as the type we wanted to receive once the __new__ function is invoked. We are talking about New function right? I don’t know other languages return type other than bound type?
And I did anther things as well:
>>> issubclass(MyClass,list)
False
>>> issubclass(MyClass,tuple)
True
>>> isinstance(a,MyClass)
False
>>> isinstance(a,tuple)
False
>>> isinstance(a,list)
True
I didn’t do more experiment because the further wasn’t bright and I decided to stop there and decided to ask StackOverflow.
The SO posts I read:
how I should structure the class using
__init__and__new__as they are different and both accepts arbitrary arguments besides default first argument.
Only rarely will you have to worry about __new__. Usually, you’ll just define __init__ and let the default __new__ pass the constructor arguments to it.
selfkeyword is in terms of name can be changed to something else? But I am wonderingclsis in terms of name is subject to change to something else as it is just a parameter name?
Both are just parameter names with no special meaning in the language. But their use is a very strong convention in the Python community; most Pythonistas will never change the names self and cls in these contexts and will be confused when someone else does.
Note that your use of def __new__(tuple) re-binds the name tuple inside the constructor function. When actually implementing __new__, you’ll want to do it as
def __new__(cls, *args, **kwargs):
# do allocation to get an object, say, obj
return obj
Albeit I said I want to return
tuple, this code works fine and returned me[1,2,3].
MyClass() will have the value that __new__ returns. There’s no implicit type checking in Python; it’s the responsibility of the programmer to return the correct type (“we’re all consenting adults here”). Being able to return a different type than requested can be useful for implementing factories: you can return a subclass of the type requested.
This also explains the issubclass/isinstance behavior you observe: the subclass relationship follows from your use of class MyClass(tuple), the isinstance reflects that you return the “wrong” type from __new__.
For reference, check out the requirements for __new__ in the Python Language Reference.
Edit: ok, here’s an example of potentially useful use of __new__. The class Eel keeps track of how many eels are alive in the process and refuses to allocate if this exceeds some maximum.
class Eel(object):
MAX_EELS = 20
n_eels = 0
def __new__(cls, *args, **kwargs):
if cls.n_eels == cls.MAX_EELS:
raise HovercraftFull()
obj = super(Eel, cls).__new__(cls)
cls.n_eels += 1
return obj
def __init__(self, voltage):
self.voltage = voltage
def __del__(self):
type(self).n_eels -= 1
def electric(self):
"""Is this an electric eel?"""
return self.voltage > 0
Mind you, there are smarter ways to accomplish this behavior.
Your question was something like,
What is the difference between
__new__and__init__in Python?
Suppose that we have a very simple class named Rectangle
class Rectangle():
def __init__(self, width:float, height:float):
self._width = width
self._hieght = height
We instantiate the Rectangle class as follows:
rocky_the_rectangle = Rectangle(10.01, 8.91)
In order to help explain what __new__ does, I will write a classmethod named make_rectangle which has almost the same behavior as the class-constructor.
class Rectangle():
def __new__(cls, *args, **kwargs):
print("__new__(" + ", ".join(str(x) for x in [cls, *args]) + ")")
obj = super().__new__(cls)
return obj
def __init__(self, width:float, height:float):
args = (width, height)
print("__init__(" + ", ".join(str(x) for x in [self, *args]) + ")")
self._width = width
self._height = height
@classmethod
def make_rectangle(cls, *args):
new_instance = cls.__new__(cls, *args)
if isinstance(new_instance, cls):
cls.__init__(new_instance, *args)
return new_instance
Now we can instantiate the Rectangle class as follows:
rocky_II = Rectangle.make_rectangle(10.01, 8.91)
The __init__ method is unable to replace the self parameter with something new and different.
For example, people often want to put a wrapper around the self parameter. You might want something like this:
import functools
import sys
class decorator:
def __new__(cls, kallable):
instance = super().__new__(cls)
instance = functools.update_wrapper(instance, kallable)
return instance
def __init__(self, kallable):
self._kallable = kallable
self._file = sys.stdout
def __call__(self, *args, **kwargs):
print("__init__(" + ", ".join(str(x) for x in [self, *args]) + ")", file=self._file)
return self._kallable(*args, **kwargs)
@decorator
def pow(base:float, exp:int):
"""
+------------------------------------------+
| EXAMPLES |
+------------------------------------------+
| BASE | EXPONENT | OUTPUT |
+------+----------+------------------------+
| 2 | 5 | 2^5 | 32 |
| 2.5 | 7 | 2.5^7 | 610.3515625 |
| 10 | 3 | 10^3 | 1000 |
| 0.1 | 5 | 0.1^5 | 0.00001 |
| 7 | 0 | 7^0 | 1 |
+------+----------+----------+-------------+
"""
base = float(base)
# convert `exp` to string to avoid flooring, or truncating, floats
exp = int(str(exp))
if exp > 0:
return base * pow(base, exp-1)
else: # exp == 2
return 1
result1 = pow(2, 5)
result2 = pow(8.1, 0)
print("pow(2, 5) == " , result1)
print("pow(8.1, 0) == ", result2)
print("What is the documentation string? The doc-string is... ", pow.__doc__)
It will not help to use functools.update_wrapper inside of __init__. If you try to write the following…
class f:
def __init__(outter, inner):
# `outter` is usually named `self`, but you are...
# ... allowed to use other names for it.
outter = functools.update_wrapper(outter, inner)
… then outter will get ignored. You cannot replace the parameter named self with a different self
__new__ allows us to replace self with a wrapper around self
class decorator:
def __new__(cls, kallable):
instance = super().__new__(cls)
instance = functools.update_wrapper(instance, kallable)
return instance
Without using functools.update_wrapper inside of __new__ the doc-string inside of the original callable will be wiped-out, ignored, not inherited, and/or shadowed.
seems that nobody had cover how the arguments are handled through the instantiation
lets build a simple class that define new and init
censured = ["Cicciogamer"]
class Foo(object):
def __new__(cls, name):
if name in censured:
print("you shouldn't do this")
return super().__new__(cls)
def __init__(self, var):
self.var = var
when you call a class object to get an instance, python implicity call
Foo.__call__(*args, **kwargs)
so with the class above you get the (maybe) undesired double arguments passing:
foo = Foo("Cicciogamer")
>>> "you shouldn't do this"
foo.var
>>> "Cicciogamer"
To achieve the control on how these parameters behave ASFAIK you must override the call method of the CLASS OBJECT not instance
you can use metaclasses but it might be overkill
class MetaFoo:
def __call__(cls, name, var):
foo_object = cls.__new__(name)
cls.__init__(foo_object, var)
return foo_object
class Foo(metaclass=MetaFoo):
...
foo = Foo("Cicciogamer", 505)
>>> "you shouldn't do this"
foo.var
>>> 505
or i think you can achieve this with simply
class Foo:
...
@classmethod
def new(cls, name, var):
foo_object = cls.__new__(name)
cls.__init__(foo_object, var)
return foo_object
__call__ = new
please let me notice if this could be achieved in a better way, or if i simply screwed up something