Using both setattr and descriptors for a python class

Question

I’m writing a python class that uses __setattr__ and __getattr__ to provide custom attribute access.

However, some attributes can’t be handled in a generic way, so I was hoping to use descriptors for those.

A problem arises in that for a descriptor, the descriptor’s __get__ will be invoked in favour of the instances __getattr__, but when assigning to an attribute, __setattr__ will be invoked in favour of the descriptors __set__.

An example:

class MyDesc(object):

    def __init__(self):
        self.val = None

    def __get__(self, instance, owner):
        print "MyDesc.__get__"
        return self.val

    def __set__(self, instance, value):
        print "MyDesc.__set__"
        self.val = value

class MyObj(object):

    foo = MyDesc()

    def __init__(self, bar):
        object.__setattr__(self, 'names', dict(
            bar=bar,
        ))
        object.__setattr__(self, 'new_names', dict())

    def __setattr__(self, name, value):
        print "MyObj.__setattr__ for %s" % name
        self.new_names[name] = value

    def __getattr__(self, name):
        print "MyObj.__getattr__ for %s" % name

        if name in self.new_names:
            return self.new_names[name]

        if name in self.names:
            return self.names[name]

        raise AttributeError(name)

if __name__ == "__main__":
    o = MyObj('bar-init')

    o.bar = 'baz'
    print o.bar

    o.foo = 'quux'
    print o.foo

prints:

MyObj.__setattr__ for bar
MyObj.__getattr__ for bar
baz
MyObj.__setattr__ for foo
MyDesc.__get__
None

The descriptor’s __set__ is never called.

Since the __setattr__ definition isn’t just overriding behaviour for a limited set of names, there’s no clear place that it can defer to object.__setattr__

Is there a recommended way to have assigning to attributes use the descriptor, if available, and __setattr__ otherwise?

Asked By: SpoonMeiser

||

Source

Answer 1

One of possible ways:

def __setattr__(self, name, value):
    print "MyObj.__setattr__ for %s" % name
    for cls in self.__class__.__mro__ + (self, ):
        if name in cls.__dict__:
            return object.__setattr__(self, name, value)
    print 'New name', name, value
    self.new_names[name] = value

It checks if name already defined in class, base classes or instance and then it calls object.__setattr__ which will execute descriptor __set__.

Another way:

def __setattr__(self, name, value):
    print "MyObj.__setattr__ for %s" % name
    try:
        object.__getattribute__(self, name)
    except AttributeError:
        print 'New name', name, value
        self.new_names[name] = value
    else:
        object.__setattr__(self, name, value)

But it will call descriptor’s __get__.

P.S.

I’m not sure about need to check all __mro__ members since MyObj will contain inherited class members in __dict__.

Maybe for cls in (self.__class__, self):... will be enough.

Answered By: reclosedev

Answer 2

update

Revisiting this answer more than 10 years later, I realized that I included a recipe, but did not explain the outstanding behavior the OP met: a descritpor’s __get__ is called before an instance’s __getattr__, yes. But the actual converse to __setattr__ is __getattribute__, not __getattr__. Inside object‘s __getattribute__ is where is the code with the search order for attributes in first place, and the code responsible for calling a descriptor’s __get__ itself. __getattr__ is called by the code in __getattribute__ itself as a last resort, after inspecting any possible descriptors, __slots__, and the instance’s __dict__.

original answer: just do what the OP wants to achieve

I think I’d approach this by having a mechanism to automatically mark which are the
descriptors in each class, and wrap the __setattr__ in a way that it’d call
object’s normal behavior for those names.

This can be easily achieved with a metaclass (and a decorator for __setattr__

def setattr_deco(setattr_func):
    def setattr_wrapper(self, attr, value):
        if attr in self._descriptors:
            return object.__setattr__(self, attr, value)
        return setattr_func(self, attr, value)
    return setattr_wrapper

class MiscSetattr(type):
    def __new__(metacls, name, bases, dct):
        descriptors = set()
        for key, obj in dct.items():
            if key == "__setattr__":
                dct[key] = setattr_deco(obj)
            elif hasattr(obj, "__get__"):
                descriptors.add(key)
        dct["_descriptors"] = descriptors
        return type.__new__(metacls, name, bases, dct)
    
# and use MiscSetattr as metaclass for your classes

Answered By: jsbueno

Using both setattr and descriptors for a python class

Question:

Answers:

Using both __setattr__ and descriptors for a python class

Question:

Answers:

Using both setattr and descriptors for a python class