Force child class to call parent method when overriding it
Question:
I am curious whether there is a way in Python to force (from the Parent class) for a parent method to be called from a child class when it is being overridden.
Example:
class Parent(object):
def __init__(self):
self.someValue=1
def start(self):
''' Force method to be called'''
self.someValue+=1
The correct implementation of what I wanted my child Child class to do would be:
class ChildCorrect(Parent):
def start(self):
Parent.start(self)
self.someValue+=1
However, is there a way to force developers that develop Child classes to specifically call(not just override) the ‘start’ method defined in the Parent class in case they forget to call it:
class ChildIncorrect(Parent):
def start(self):
'''Parent method not called, but correctly overridden'''
self.someValue+=1
Also, if this is not considered best practice, what would be an alternative?
Answers:
All you need is super.
With it, your code could look like this:
class Parent(object):
def __init__(self):
self.someValue = 1
def start(self):
self.someValue += 1
class Child(Parent):
def start(self):
# put code here
super().start()
# or here
However, is there a way to force developers that develop Child classes
to specifically call(not just override) the ‘start’ method defined in
the Parent class in case they forget to call it
In Python you can not control how others override your functions.
Very little in Python is about forcing other programmers to code a certain way. Even in the standard library you will find warnings such as this:
If the subclass overrides the constructor, it must make sure to invoke the base class constructor before doing anything else to the thread.
While it would be possible to use a metaclass to modify subclasses’ behavior, the best it could reliably do in this case would be to replace the new methods with yet another method that would take care of calling both the replaced method and the original method. However, that would have problems if the programmer then did call the parent method in the new code, plus it would lead to bad habits of not calling parent methods when one should.
In other words, Python is not built that way.
With some metaclass hacks, you could detect at runtime whether the parent start has been called or not. However I will certainly NOT recommand using this code in real situations, it’s probably buggy in many ways and it’s not pythonic to enforce such restrictions.
class ParentMetaClass(type):
parent_called = False
def __new__(cls, name, bases, attrs):
print cls, name, bases, attrs
original_start = attrs.get('start')
if original_start:
if name == 'Parent':
def patched_start(*args, **kwargs):
original_start(*args, **kwargs)
cls.parent_called = True
else:
def patched_start(*args, **kwargs):
original_start(*args, **kwargs)
if not cls.parent_called:
raise ValueError('Parent start not called')
cls.parent_called = False
attrs['start'] = patched_start
return super(ParentMetaClass, cls).__new__(cls, name, bases, attrs)
class Parent(object):
__metaclass__ = ParentMetaClass
def start(self):
print 'Parent start called.'
class GoodChild(Parent):
def start(self):
super(GoodChild, self).start()
print 'I am a good child.'
class BadChild(Parent):
def start(self):
print 'I am a bad child, I will raise a ValueError.'
How (not) to do it
No, there is no safe way to force users to call super. Let us go over a few options which would reach that or a similar goal and discuss why it’s a bad idea. In the next section, I will also discuss what the sensible (with respect to the Python community) way to deal with the situation is.
-
A metaclass could check, at the time the subclass is defined, whether a method overriding the target method (= has the same name as the target method) calls super with the appropriate arguments.
This requires deeply implementation-specific behaviour, such as using the dis module of CPython. There are no circumstances where I could imagine this to be a good idea—it is dependent on the specific version of CPython and on the fact that you are using CPython at all.
-
A metaclass could co-operate with the baseclass. In this scenario, the baseclass notifies the metaclass when the inherited method is called and the metaclass wraps any overriding methods with a piece of guard code.
The guard code tests whether the inherited method got called during the execution of the overriding method. This has the downside that a separate notification channel for each method which needs this “feature” is required, and in addition thread-safety and re-entrance would be a concern. Also, the overriding method has finished execution at the point you notice that it has not called the inherited method, which may be bad depending on your scenario.
It also opens the question: What to do if the overriding method has not called the inherited method? Raising an exception might be unexpected by the code using the method (or, even worse, it might assume that the method had no effect at all, which is not true).
Also the late feedback the developer overriding the class (if they get that feedback at all!) is bad.
-
A metaclass could generate a piece of guard code for every overriden method which calls the inherited method automatically before or after the overriding method has executed.
The downside here is developers will not be expecting that the inherited method is being called automatically and have no way to stop that call (e.g. if a precondition special to the subclass is not met) or control when in their own overriding method the inherited method is called.
This violates a good bunch of sensible principles when coding python (avoid surprises, explicit is better than implicit, and possibly more).
-
A combination of point 2 and 3. Using the co-operation of base- and metaclass from point 2, the guard code from point 3 could be extended to automatically call super iff the overriding method has not called super themselves.
Again, this is unexpected, but it resolves the issues with duplicate super calls and how to handle a method which does not call super.
However, there are still remaining problems. While thread-safety could be fixed with thread-locals, there is still no way for an overriding method to abort the call to super when a precondition is not met, except by raising an exception which may not be desirable in all cases. Also, super can only be called automatically after the overriding method, not before, which, again, in some scenarios is undesirable.
Also, none of this help against re-binding the attribute during the lifetime of the object and class, although this can be helped by using descriptors and/or extending the metaclass to take care of it.
Why not to do it
Also, if this is not considered best practice, what would be an alternative?
A common best-practice with Python is to assume that you are among consenting adults. That means, that noone is actively trying to do nasty things to your code, unless you allow them to. In such an ecosystem, it would make sense to tack a .. warning:: in the documentation of the method or class, so that anyone inheriting from that class knows what they have to do.
Also, calling the super method at an appropriate point makes sense in so many contexts that developers using the base class will consider it anyways and only forget about it accidentally. In such a case, using the metaclass from the third point above would not help either—users would have to remember not to call super, which might be an issue by itself, especially for experienced programmers.
It also violates the principle of least surprise and “explicit is better than implicit” (noone expects the inherited method to be called implicitly!). This, again, would have to be documented well, in which case you can also resort to just not have super be called automatically and just document that it makes even more sense than usual to call the inherited method.
If the class hierarchy is under your control, you can use what the Gang of Four (Gamma, et al) Design Patterns book calls the Template Method Pattern:
class MyBase:
def MyMethod(self):
# place any code that must always be called here...
print "Base class pre-code"
# call an internal method that contains the subclass-specific code
self._DoMyMethod()
# ...and then any additional code that should always be performed
# here.
print "base class post-code!"
def _DoMyMethod(self):
print "BASE!"
class MyDerived(MyBase):
def _DoMyMethod(self):
print "DERIVED!"
b = MyBase()
d = MyDerived()
b.MyMethod()
d.MyMethod()
outputs:
Base class pre-code
BASE!
base class post-code!
Base class pre-code
DERIVED!
base class post-code!
This does not relate directly to Python, it would be the case for most OO languages. You should define an abstract method in your base class, make it non-public (but available for overriding by subclasses) and also create a public template method in the parent class, that will call this abstract method (defined in your subclass). The client code will not be able to call the method in question directly, but will be able to call the template method which, in turn, will make sure that the method in question is called in the correct order/context.
In other words, your system must not depend on the subclasses calling parent’s implementation. If that is a must, then you indeed should review your implementation and decompose the method in question (eg, use the template method pattern).
Also, some languages (like Java) implicitly call the parent’s constructor, if the subclass’es constructor does not do it explicitly (does not work for plain methods though).
In any case, please never rely on “hacks”. that is not the right way to solve problems.
I think @bgporter was on the right track, but it is missing syntax enforceability which the following pattern helps accomplish:
from abc import abstractmethod
class Parent:
def __init__(self):
self.value = 1
def before_start(self):
self.value += 1
@abstractmethod
def on_start(self):
...
def after_start(self):
pass
def start(self):
"""
Please don't override this. Thank you.
"""
self.before_start()
self.on_start()
self.after_start()
class Child(Parent):
"""
mypy, pylance and and pycharm will warn about
unimplemented abstract method if on_start is missing.
Child can now override before_start and after_start behaviors as well.
This pattern is often seen in state machines so perhaps that's what you are after.
"""
def on_start(self):
self.value += 1
child = Child()
child.start()
print(child.value) # prints 3
I am curious whether there is a way in Python to force (from the Parent class) for a parent method to be called from a child class when it is being overridden.
Example:
class Parent(object):
def __init__(self):
self.someValue=1
def start(self):
''' Force method to be called'''
self.someValue+=1
The correct implementation of what I wanted my child Child class to do would be:
class ChildCorrect(Parent):
def start(self):
Parent.start(self)
self.someValue+=1
However, is there a way to force developers that develop Child classes to specifically call(not just override) the ‘start’ method defined in the Parent class in case they forget to call it:
class ChildIncorrect(Parent):
def start(self):
'''Parent method not called, but correctly overridden'''
self.someValue+=1
Also, if this is not considered best practice, what would be an alternative?
All you need is super.
With it, your code could look like this:
class Parent(object):
def __init__(self):
self.someValue = 1
def start(self):
self.someValue += 1
class Child(Parent):
def start(self):
# put code here
super().start()
# or here
However, is there a way to force developers that develop Child classes
to specifically call(not just override) the ‘start’ method defined in
the Parent class in case they forget to call it
In Python you can not control how others override your functions.
Very little in Python is about forcing other programmers to code a certain way. Even in the standard library you will find warnings such as this:
If the subclass overrides the constructor, it must make sure to invoke the base class constructor before doing anything else to the thread.
While it would be possible to use a metaclass to modify subclasses’ behavior, the best it could reliably do in this case would be to replace the new methods with yet another method that would take care of calling both the replaced method and the original method. However, that would have problems if the programmer then did call the parent method in the new code, plus it would lead to bad habits of not calling parent methods when one should.
In other words, Python is not built that way.
With some metaclass hacks, you could detect at runtime whether the parent start has been called or not. However I will certainly NOT recommand using this code in real situations, it’s probably buggy in many ways and it’s not pythonic to enforce such restrictions.
class ParentMetaClass(type):
parent_called = False
def __new__(cls, name, bases, attrs):
print cls, name, bases, attrs
original_start = attrs.get('start')
if original_start:
if name == 'Parent':
def patched_start(*args, **kwargs):
original_start(*args, **kwargs)
cls.parent_called = True
else:
def patched_start(*args, **kwargs):
original_start(*args, **kwargs)
if not cls.parent_called:
raise ValueError('Parent start not called')
cls.parent_called = False
attrs['start'] = patched_start
return super(ParentMetaClass, cls).__new__(cls, name, bases, attrs)
class Parent(object):
__metaclass__ = ParentMetaClass
def start(self):
print 'Parent start called.'
class GoodChild(Parent):
def start(self):
super(GoodChild, self).start()
print 'I am a good child.'
class BadChild(Parent):
def start(self):
print 'I am a bad child, I will raise a ValueError.'
How (not) to do it
No, there is no safe way to force users to call super. Let us go over a few options which would reach that or a similar goal and discuss why it’s a bad idea. In the next section, I will also discuss what the sensible (with respect to the Python community) way to deal with the situation is.
-
A metaclass could check, at the time the subclass is defined, whether a method overriding the target method (= has the same name as the target method) calls super with the appropriate arguments.
This requires deeply implementation-specific behaviour, such as using the
dismodule of CPython. There are no circumstances where I could imagine this to be a good idea—it is dependent on the specific version of CPython and on the fact that you are using CPython at all. -
A metaclass could co-operate with the baseclass. In this scenario, the baseclass notifies the metaclass when the inherited method is called and the metaclass wraps any overriding methods with a piece of guard code.
The guard code tests whether the inherited method got called during the execution of the overriding method. This has the downside that a separate notification channel for each method which needs this “feature” is required, and in addition thread-safety and re-entrance would be a concern. Also, the overriding method has finished execution at the point you notice that it has not called the inherited method, which may be bad depending on your scenario.
It also opens the question: What to do if the overriding method has not called the inherited method? Raising an exception might be unexpected by the code using the method (or, even worse, it might assume that the method had no effect at all, which is not true).
Also the late feedback the developer overriding the class (if they get that feedback at all!) is bad.
-
A metaclass could generate a piece of guard code for every overriden method which calls the inherited method automatically before or after the overriding method has executed.
The downside here is developers will not be expecting that the inherited method is being called automatically and have no way to stop that call (e.g. if a precondition special to the subclass is not met) or control when in their own overriding method the inherited method is called.
This violates a good bunch of sensible principles when coding python (avoid surprises, explicit is better than implicit, and possibly more).
-
A combination of point 2 and 3. Using the co-operation of base- and metaclass from point 2, the guard code from point 3 could be extended to automatically call super iff the overriding method has not called super themselves.
Again, this is unexpected, but it resolves the issues with duplicate super calls and how to handle a method which does not call super.
However, there are still remaining problems. While thread-safety could be fixed with thread-locals, there is still no way for an overriding method to abort the call to super when a precondition is not met, except by raising an exception which may not be desirable in all cases. Also, super can only be called automatically after the overriding method, not before, which, again, in some scenarios is undesirable.
Also, none of this help against re-binding the attribute during the lifetime of the object and class, although this can be helped by using descriptors and/or extending the metaclass to take care of it.
Why not to do it
Also, if this is not considered best practice, what would be an alternative?
A common best-practice with Python is to assume that you are among consenting adults. That means, that noone is actively trying to do nasty things to your code, unless you allow them to. In such an ecosystem, it would make sense to tack a .. warning:: in the documentation of the method or class, so that anyone inheriting from that class knows what they have to do.
Also, calling the super method at an appropriate point makes sense in so many contexts that developers using the base class will consider it anyways and only forget about it accidentally. In such a case, using the metaclass from the third point above would not help either—users would have to remember not to call super, which might be an issue by itself, especially for experienced programmers.
It also violates the principle of least surprise and “explicit is better than implicit” (noone expects the inherited method to be called implicitly!). This, again, would have to be documented well, in which case you can also resort to just not have super be called automatically and just document that it makes even more sense than usual to call the inherited method.
If the class hierarchy is under your control, you can use what the Gang of Four (Gamma, et al) Design Patterns book calls the Template Method Pattern:
class MyBase:
def MyMethod(self):
# place any code that must always be called here...
print "Base class pre-code"
# call an internal method that contains the subclass-specific code
self._DoMyMethod()
# ...and then any additional code that should always be performed
# here.
print "base class post-code!"
def _DoMyMethod(self):
print "BASE!"
class MyDerived(MyBase):
def _DoMyMethod(self):
print "DERIVED!"
b = MyBase()
d = MyDerived()
b.MyMethod()
d.MyMethod()
outputs:
Base class pre-code
BASE!
base class post-code!
Base class pre-code
DERIVED!
base class post-code!
This does not relate directly to Python, it would be the case for most OO languages. You should define an abstract method in your base class, make it non-public (but available for overriding by subclasses) and also create a public template method in the parent class, that will call this abstract method (defined in your subclass). The client code will not be able to call the method in question directly, but will be able to call the template method which, in turn, will make sure that the method in question is called in the correct order/context.
In other words, your system must not depend on the subclasses calling parent’s implementation. If that is a must, then you indeed should review your implementation and decompose the method in question (eg, use the template method pattern).
Also, some languages (like Java) implicitly call the parent’s constructor, if the subclass’es constructor does not do it explicitly (does not work for plain methods though).
In any case, please never rely on “hacks”. that is not the right way to solve problems.
I think @bgporter was on the right track, but it is missing syntax enforceability which the following pattern helps accomplish:
from abc import abstractmethod
class Parent:
def __init__(self):
self.value = 1
def before_start(self):
self.value += 1
@abstractmethod
def on_start(self):
...
def after_start(self):
pass
def start(self):
"""
Please don't override this. Thank you.
"""
self.before_start()
self.on_start()
self.after_start()
class Child(Parent):
"""
mypy, pylance and and pycharm will warn about
unimplemented abstract method if on_start is missing.
Child can now override before_start and after_start behaviors as well.
This pattern is often seen in state machines so perhaps that's what you are after.
"""
def on_start(self):
self.value += 1
child = Child()
child.start()
print(child.value) # prints 3