AttributeErrors: undesired interaction between @property and __getattr__

Question:

I have a problem with AttributeErrors raised in a @property in combination with __getattr__() in python:

Example code:

>>> def deeply_nested_factory_fn():
...     a = 2
...     return a.invalid_attr
...
>>> class Test(object):
...     def __getattr__(self, name):
...         if name == 'abc':
...             return 'abc'
...         raise AttributeError("'Test' object has no attribute '%s'" % name)
...     @property
...     def my_prop(self):
...         return deeply_nested_factory_fn()
...
>>> test = Test()
>>> test.my_prop
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in __getattr__
AttributeError: 'Test' object has no attribute 'my_prop'

In my case, this is a highly misleading error message, because it hides the fact that deeply_nested_factory_fn() has a mistake.

Based on the idea in Tadhg McDonald-Jensen’s answer, my currently best solution is the following. Any hints on how to get rid of the __main__. prefix to AttributeError and the reference to attributeErrorCatcher in the traceback would be much appreciated.

>>> def catchAttributeErrors(func):
...     AttributeError_org = AttributeError
...     def attributeErrorCatcher(*args, **kwargs):
...         try:
...             return func(*args, **kwargs)
...         except AttributeError_org as e:
...             import sys
...             class AttributeError(Exception):
...                 pass
...             etype, value, tb = sys.exc_info()
...             raise AttributeError(e).with_traceback(tb.tb_next) from None
...     return attributeErrorCatcher
...
>>> def deeply_nested_factory_fn():
...     a = 2
...     return a.invalid_attr
...
>>> class Test(object):
...     def __getattr__(self, name):
...         if name == 'abc':
...             # computing come other attributes
...             return 'abc'
...         raise AttributeError("'Test' object has no attribute '%s'" % name)
...     @property
...     @catchAttributeErrors
...     def my_prop(self):
...         return deeply_nested_factory_fn()
...
>>> class Test1(object):
...     def __init__(self):
...         test = Test()
...         test.my_prop
...
>>> test1 = Test1()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in __init__
  File "<stdin>", line 11, in attributeErrorCatcher
  File "<stdin>", line 10, in my_prop
  File "<stdin>", line 3, in deeply_nested_factory_fn
__main__.AttributeError: 'int' object has no attribute 'invalid_attr'
Asked By: ARF

||
Source

Answers:

If you’re willing to exclusively use new-style classes, you could overload __getattribute__ instead of __getattr__:

class Test(object):
    def __getattribute__(self, name):
        if name == 'abc':
            return 'abc'
        else:
            return object.__getattribute__(self, name)
    @property
    def my_prop(self):
        return deeply_nested_factory_fn()

Now your stack trace will properly mention deeply_nested_factory_fn.

Traceback (most recent call last):
  File "C:pythonmyprogram.py", line 16, in <module>
    test.my_prop
  File "C:pythonmyprogram.py", line 10, in __getattribute__
    return object.__getattribute__(self, name)
  File "C:pythonmyprogram.py", line 13, in my_prop
    return deeply_nested_factory_fn()
  File "C:pythonmyprogram.py", line 3, in deeply_nested_factory_fn
    return a.invalid_attr
AttributeError: 'int' object has no attribute 'invalid_attr'
Answered By: Kevin

You can create a custom Exception that appears to be an AttributeError but will not trigger __getattr__ since it is not actually an AttributeError.

UPDATED: the traceback message is greatly improved by reassigning the .__traceback__ attribute before re-raising the error:

class AttributeError_alt(Exception):
    @classmethod
    def wrapper(err_type, f):
        """wraps a function to reraise an AttributeError as the alternate type"""
        @functools.wraps(f)
        def alt_AttrError_wrapper(*args,**kw):
            try:
                return f(*args,**kw)
            except AttributeError as e:
                new_err = err_type(e)
                new_err.__traceback__ = e.__traceback__.tb_next
                raise new_err from None
        return alt_AttrError_wrapper

Then when you define your property as:

@property
@AttributeError_alt.wrapper
def my_prop(self):
    return deeply_nested_factory_fn()

and the error message you will get will look like this:

Traceback (most recent call last):
  File ".../test.py", line 34, in <module>
    test.my_prop
  File ".../test.py", line 14, in alt_AttrError_wrapper
    raise new_err from None
  File ".../test.py", line 30, in my_prop
    return deeply_nested_factory_fn()
  File ".../test.py", line 20, in deeply_nested_factory_fn
    return a.invalid_attr
AttributeError_alt: 'int' object has no attribute 'invalid_attr'

notice there is a line for raise new_err from None but it is above the lines from within the property call. There would also be a line for return f(*args,**kw) but that is omitted with .tb_next.

I am fairly sure the best solution to your problem has already been suggested and you can see the previous revision of my answer for why I think it is the best option. Although honestly if there is an error that is incorrectly being suppressed then raise a bloody RuntimeError chained to the one that would be hidden otherwise:

def assert_no_AttributeError(f):
    @functools.wraps(f)
    def assert_no_AttrError_wrapper(*args,**kw):
        try:
            return f(*args,**kw)
        except AttributeError as e:
            e.__traceback__ = e.__traceback__.tb_next
            raise RuntimeError("AttributeError was incorrectly raised") from e
    return assert_no_AttrError_wrapper

then if you decorate your property with this you will get an error like this:

Traceback (most recent call last):
  File ".../test.py", line 27, in my_prop
    return deeply_nested_factory_fn()
  File ".../test.py", line 17, in deeply_nested_factory_fn
    return a.invalid_attr
AttributeError: 'int' object has no attribute 'invalid_attr'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".../test.py", line 32, in <module>
    x.my_prop
  File ".../test.py", line 11, in assert_no_AttrError_wrapper
    raise RuntimeError("AttributeError was incorrectly raised") from e
RuntimeError: AttributeError was incorrectly raised

Although if you expect more then just one thing to raise an AttributeError then you might want to just overload __getattribute__ to check for any peculiar error for all lookups:

def __getattribute__(self,attr):
    try:
        return object.__getattribute__(self,attr)
    except AttributeError as e:
        if str(e) == "{0.__class__.__name__!r} object has no attribute {1!r}".format(self,attr):
            raise #normal case of "attribute not found"
        else: #if the error message was anything else then it *causes* a RuntimeError
            raise RuntimeError("Unexpected AttributeError") from e

This way when something goes wrong that you are not expecting you will know it right away!

Answered By: Tadhg McDonald-Jensen

Just in case others find this: the problem with the example on top is that an AttributeError is raised inside __getattr__. Instead, one should call self.__getattribute__(attr) to let that raise.

Example

def deeply_nested_factory_fn():
    a = 2
    return a.invalid_attr

class Test(object):
    def __getattr__(self, name):
        if name == 'abc':
            return 'abc'
        return self.__getattribute__(name)
    @property
    def my_prop(self):
        return deeply_nested_factory_fn()

test = Test()
test.my_prop

This yields

AttributeError                            Traceback (most recent call last)
Cell In [1], line 15
     12         return deeply_nested_factory_fn()
     14 test = Test()
---> 15 test.my_prop

Cell In [1], line 9, in Test.__getattr__(self, name)
      7 if name == 'abc':
      8     return 'abc'
----> 9 return self.__getattribute__(name)

Cell In [1], line 12, in Test.my_prop(self)
     10 @property
     11 def my_prop(self):
---> 12     return deeply_nested_factory_fn()

Cell In [1], line 3, in deeply_nested_factory_fn()
      1 def deeply_nested_factory_fn():
      2     a = 2
----> 3     return a.invalid_attr

AttributeError: 'int' object has no attribute 'invalid_attr'
Answered By: user2483412