Python: replacing all appearances of a class constructor in superclass from subclass

Question

This is probably a stupid question. But I encountered this need when writing a tool that needs to slightly modify the API from an existing package. The minimal example is as such

class Foo:
    def __init__(self):
        print("foo")        
    
class Qux(Foo):
    def __init__(self):
        print("qux")
        
class Baz:
    def __init__(self):
        self.foo = Foo()
        
    def f(self):
        # ... a lot of code
        Foo()
        # ... a lot of code
        
    def g(self):
        # ... a lot of code
        Foo()
        # ... a lot of code

class Bar(Baz):
    Baz.Foo = Qux #（my rough idea but it doesn't work)
    

b = Bar()
b.f()
b.g()
# want to print qux all above

I want to achieve this without overloading all methods from Baz that used the Foo constructor to use Qux instead. Is there any way to do this without touching the code for the superclass?

Also, please kindly enlighten me if there’s a name for such situation. I tried to search up solution on Stackoverflow but had no luck. Thanks!

Asked By: Macrophage

||

Source

Answer 1

What you need there is called "monkey patching" – an operation possible in dynamic languages such as Javascript, Python and Ruby, in which either a module level variable or an attribute of a class/instance is replaced by other at runtime.

It is popular with testing, as it allows one to replace functions and classes with mocks – but if used with wisdom, can be made to work in production code.

Its usage can range from simple to complicated, depending on how things are arranged there:

If you can change all occurrences and usages of Bar in the process to use Qux instead of Foo, and never look back – you are in luck for the simplest scenario, with no need for a Baz subclass, and no need to rollback the monkey-patching.

Let’s suppose the usages of Foo in Bar code are just as shown in your example: the code assumes that there is a Foo name bound at module level (the module where Bar is declared), and just do Foo(), and not, in contrast, module1.Foo() – you just have to replace that name before instantiating any Bar objects.

If Bar is defined in module2, for example, you can write at the very beggining of your Python project (usually the __init__.py file at the project base, or, sometimes an app.py file for stuff that will run in containers) you do:

import module2
from module3 import Qux

module2.Foo = Qux

# program follows as usual
(...)
from module2 import Bar
def myfun():
    x = Bar() # <- when this is run, `Bar` will actually "see" Qux insteaf of Foo

That was the simple thing. Now, there are intermediate scenarios, like Foo sometimes showing up as Foo and sometimes as module1.Foo in the source code, or Foo bein used in other places than in the Bar class – there are ways to adapt the idea above to handle that.

using the `Baz` class approach

But if you really need a different Baz class, then we have to fine tune the change – and it won’t be a proper "monkey patching" anymore. The original Bar class would run pointing to Foo, and Baz is a new, isolated class – as you ask for it. But it does not come down as simple inheritance — it will require you to clone the Bar class into a new Baz. It can optionally be made to inherit from Bar if it is needed to pass isinstance checks, though it will override all methods in the class with clones of those methods – and if there are uses of "super()" call in methods that also call Foo() it would require yet another level of hacking around.

I won’t worry about fixing super calls right now – if you need these calls to work, please mention in the comments.

So the steps to be taken are:

duplicate the Bar class by calling type with the new name, Bar as a base, and
- iterating over all members of Bar and for each method, clone the method.
- this clonning however should recreate the function objects, replacing the its __globals__ attribute with a copy where Foo points to Qux.

Actually, "super()" calls will work straightforward – I found that out when trying to write some special code for it to work, it turns out it is the default behavior: one will want to pass straight from the method in the cloned class to the method of the superclass of the original class (a method in Bar that uses super() will land directly in the superclass to Baz, skipping the original method in Baz, that is)

def replace_inner(base_cls, new_name, replacements):
    from types import FunctionType
    class New(base_cls):
        pass
    New.__name__ = new_name
    new_globals = None
    for name, obj in base_cls.__dict__.items():
        if not isinstance(obj, FunctionType):
            try:
                setattr(New, name, obj)
            except AttributeError:
                pass
            continue
        if not new_globals:
            new_globals = copy(obj.__globals__)
            new_globals.update(replacements)
        new_func = FunctionType(obj.__code__, new_globals, obj.__name__, obj.__defaults__, obj.__closure__)
        setattr(New, name, new_func)
    return New

And this working in the interactive mode (Foo and Qux are defined as in your example)

In [52]: class A:
    ...:     def __init__(self):
    ...:         print("A")
    ...:

In [53]: class B(A):
    ...:     def __init__(self):
    ...:         self.f = Foo()
    ...:         super().__init__()
    ...:

In [54]: C = replace_inner(B, "C", {"Foo": Qux})

In [55]: c = C()
Qux
A

In [56]: c.f
Out[56]: <__main__.Qux at 0x7f0f399ac310>

Answered By: jsbueno

Python: replacing all appearances of a class constructor in superclass from subclass

Question:

Answers:

using the `Baz` class approach

Python: replacing all appearances of a class constructor in superclass from subclass

Question:

Answers:

using the Baz class approach

using the `Baz` class approach