How to use pyfakefs in conjunction with importlib.resources.path

Question

Given some installed package, the following code can be used to print its location on the filesystem:

import importlib.resources

def print_package_path(pkg):
    with importlib.resources.path(pkg, "") as path:
        print(path)

print_package_path("importlib")
# /home/me/python_3.8/lib/python3.8/importlib on my machine

If I want to test a function that contains such a statement with pytest as the test suite and pyfakefs to fake the filesystem during the test, it will crash with a confusing error (IsADirectoryError: [Errno 21] Is a directory on ubuntu and PermissionError: [Errno 13] Permission denied on windows) – it’s not even necessary to do anything with the fs fixture, it just needs to be part of the test function’s signature, e.g.

def test_package_path(fs):
    print_package_path("importlib")

will result in

______________________________ test_package_path _______________________________

fs = <pyfakefs.fake_filesystem.FakeFilesystem object at 0x7f84f2996910>

    def test_package_path(fs):
>       print_package_path()

/home/me/foo.py:11: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/home/me/foo.py:4: in print_package_path
    with importlib.resources.path("importlib", "") as path:
/home/me/pythons/python_3.8/lib/python3.8/contextlib.py:113: in __enter__
    return next(self.gen)
/home/me/venv/lib/python3.8/importlib/resources.py:201: in path
    with open_binary(package, resource) as fp:
/home/me/venv/lib/python3.8/importlib/resources.py:91: in open_binary
    return reader.open_resource(resource)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_frozen_importlib_external.SourceFileLoader object at 0x7f84f7527370>
resource = ''

>   ???
E   IsADirectoryError: [Errno 21] Is a directory: '/home/me/venv/lib/python3.8/importlib'

<frozen importlib._bootstrap_external>:988: IsADirectoryError

which makes zero sense to me. Should a directory never have been usable as a resource in the first place?

I must admit I don’t really understand the importlib.resources module, so I might just be using it wrong (my actual use case is creating a file during development, and avoiding the use of __file__ to find the right location).

Asked By: Arne

||

Source

Answer 1

See pyfakefs documentation that states:

Modules may not work with pyfakefs for several reasons. pyfakefs works by patching some file system related modules and functions, specifically:

most file system related functions in the os and os.path modules

the pathlib module

the build-in open function and io.open

shutil.disk_usage

Simply including fs fixture enables patching these modules. If you really need to use pyfakefs, either provide everything your code expects (even indirectly) or start your tests with paused fs and enable it only to test specific things that can’t be tested otherwise. In this case it’s io.open that breaks.

Providing the expected files works by calling fs.add_real_directory before executing the functions that rely on the existence of the files, like this:

def test_package_path(fs):
    fs.add_real_directory(os.path.dirname(importlib.__file__))
    print_package_path()

, where importlib.__file__ needs to be replaced with the full path of whatever is accessed by importlib.resources.path in the tested code. This method is also safe against file creation in the tested function, since the fake fs is aware of all changes and never applies them to the actual files:

The access to the files is by default read-only, but even if you add them using read_only=False, the files are written only in the fake system (e.g. in memory). The real files are never changed.

Answered By: zgoda

How to use pyfakefs in conjunction with importlib.resources.path

Question:

Answers: