Module's getattr is called twice

Question

PEP-562 introduced __getattr__ for modules. While testing I noticed this magic method is called twice when called in this form: from X import Y.

file_b.py:

def __getattr__(name):
    print("__getattr__ called:", name)

file_a.py:

from file_b import foo, bar

output:

__getattr__ called: __path__
__getattr__ called: foo
__getattr__ called: bar
__getattr__ called: foo
__getattr__ called: bar

I run it with: python file_a.py. The interpreter version is: 3.10.6

Could you please let me know the reason behind this?

Asked By: S.B

||

Source

Answer 1

Because your __getattr__ returns None for __path__ instead of raising an AttributeError, the import machinery thinks your file_b is a package. None isn’t actually a valid __path__, but all __import__ checks here is hasattr(module, '__path__'). It doesn’t check the value:

elif hasattr(module, '__path__'):
    return _handle_fromlist(module, fromlist, _gcd_import)

Because the import machinery thinks your file_b is a package, it calls _handle_fromlist, the function responsible for loading package submodules specified in a from import. _handle_fromlist will go through all the names you specified, and for each one, it will call hasattr to check whether an attribute with that name exists on file_b. If an attribute is not found, _handle_fromlist will attempt to import a submodule with the given name:

for x in fromlist:
    if not isinstance(x, str):
        if recursive:
            where = module.__name__ + '.__all__'
        else:
            where = "``from list''"
        raise TypeError(f"Item in {where} must be str, "
                        f"not {type(x).__name__}")
    elif x == '*':
        if not recursive and hasattr(module, '__all__'):
            _handle_fromlist(module, module.__all__, import_,
                             recursive=True)
    elif not hasattr(module, x):
        from_name = '{}.{}'.format(module.__name__, x)
        try:
            _call_with_frames_removed(import_, from_name)
        except ModuleNotFoundError as exc:
            # Backwards-compatibility dictates we ignore failed
            # imports triggered by fromlist for modules that don't
            # exist.
            if (exc.name == from_name and
                sys.modules.get(from_name, _NEEDS_LOADING) is not None):
                continue
            raise

This is where the first __getattr__ calls come from: hasattr attempts to get the foo and bar attributes on file_b, triggering your __getattr__.

The second __getattr__ calls are just the calls you would expect, retrieving the foo and bar attributes you said to import so the import statement can assign them to names in the namespace the import was executed in.

Answered By: user2357112

Module's getattr is called twice

Question:

Answers:

Module's __getattr__ is called twice

Question:

Answers:

Module's getattr is called twice