How can I intercept calls to python's "magic" methods in new style classes?

Question

I’m trying to intercept calls to python’s double underscore magic methods in new style classes. This is a trivial example but it show’s the intent:

class ShowMeList(object):
    def __init__(self, it):
        self._data = list(it)

    def __getattr__(self, name):
        attr = object.__getattribute__(self._data, name)
        if callable(attr):
            def wrapper(*a, **kw):
                print "before the call"
                result = attr(*a, **kw)
                print "after the call"
                return result
            return wrapper
        return attr

If I use that proxy object around list I get the expected behavior for non-magic methods but my wrapper function is never called for magic methods.

>>> l = ShowMeList(range(8))

>>> l #call to __repr__
<__main__.ShowMeList object at 0x9640eac>

>>> l.append(9)
before the call
after the call

>> len(l._data)
9

If I don’t inherit from object (first line class ShowMeList:) everything works as expected:

>>> l = ShowMeList(range(8))

>>> l #call to __repr__
before the call
after the call
[0, 1, 2, 3, 4, 5, 6, 7]

>>> l.append(9)
before the call
after the call

>> len(l._data)
9

How do I accomplish this intercept with new style classes?

Asked By: Finn

||

Source

Answer 1

As of the answers to Asymmetric behavior for __getattr__, newstyle vs oldstyle classes (see also the Python docs), modifying access to “magic” methods with __getattr__ or __getattribute__ is just not possible with new-style classes. This restriction makes the interpreter much faster.

Answered By: Jan Pöschko

Answer 2

Using __getattr__ and __getattribute__ are the last resources of a class to respond to getting an attribute.

Consider the following:

>>> class C:
    x = 1
    def __init__(self):
        self.y = 2
    def __getattr__(self, attr):
        print(attr)

>>> c = C()
>>> c.x
1
>>> c.y
2
>>> c.z
z

The __getattr__ method is only called when nothing else works (It will not work on operators, and you can read about that here).

On your example, the __repr__ and many other magic methods are already defined in the object class.

One thing can be done, thought, and it is to define those magic methods and make then call the __getattr__ method. Check this other question by me and its answers (link) to see some code doing that.

Answered By: JBernardo

Answer 3

Cut and copy from the documentation:

For old-style classes, special methods are always looked up in exactly the same way as any other method or attribute.

For new-style classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary.

Answered By: mg.

Answer 4

For performance reasons, Python always looks in the class (and parent classes’) __dict__ for magic methods and does not use the normal attribute lookup mechanism. A workaround is to use a metaclass to automatically add proxies for magic methods at the time of class creation; I’ve used this technique to avoid having to write boilerplate call-through methods for wrapper classes, for example.

class Wrapper(object):
    """Wrapper class that provides proxy access to some internal instance."""

    __wraps__  = None
    __ignore__ = "class mro new init setattr getattr getattribute"

    def __init__(self, obj):
        if self.__wraps__ is None:
            raise TypeError("base class Wrapper may not be instantiated")
        elif isinstance(obj, self.__wraps__):
            self._obj = obj
        else:
            raise ValueError("wrapped object must be of %s" % self.__wraps__)

    # provide proxy access to regular attributes of wrapped object
    def __getattr__(self, name):
        return getattr(self._obj, name)
    
    # create proxies for wrapped object's double-underscore attributes
    class __metaclass__(type):
        def __init__(cls, name, bases, dct):

            def make_proxy(name):
                def proxy(self, *args):
                    return getattr(self._obj, name)
                return proxy

            type.__init__(cls, name, bases, dct)
            if cls.__wraps__:
                ignore = set("__%s__" % n for n in cls.__ignore__.split())
                for name in dir(cls.__wraps__):
                    if name.startswith("__"):
                        if name not in ignore and name not in dct:
                            setattr(cls, name, property(make_proxy(name)))

Usage:

class DictWrapper(Wrapper):
    __wraps__ = dict

wrapped_dict = DictWrapper(dict(a=1, b=2, c=3))

# make sure it worked....
assert "b" in wrapped_dict                        # __contains__
assert wrapped_dict == dict(a=1, b=2, c=3)        # __eq__
assert "'a': 1" in str(wrapped_dict)              # __str__
assert wrapped_dict.__doc__.startswith("dict()")  # __doc__

Answered By: kindall

How can I intercept calls to python's "magic" methods in new style classes?

Question:

Answers: