How to mock os.listdir to pretend files and directories in Python?

Question

I have a proprietary repository format and I’m trying to develop a Python module to process these repositories. Repo format goes as:

/home/X/
       |
       + alpha/
       |
       + beta/
       |
       + project.conf

Here, X is a project. alpha and beta are folders inside this project and they represent groups in this project. A group is a container in this repo and what it represents is really not relevant for this question. The repo X also has files in its root level; project.conf is an example of such a file.

I have a class called Project that abstracts projects such as X. The Project class has a method load() that builds an in-memory representation.

class Project(object):

    def load(self):
        for entry in os.listdir(self.root):
            path = os.path.join(self.root, entry)
            if os.path.isdir(path):
                group = Group(path)
                self.groups.append(group)
                group.load()
            else:
                # process files ...

To unit test the load() method by mocking the file system, I have:

import unittest
from unittest import mock
import Project

class TestRepo(unittest.TestCase):

    def test_load_project(self):
        project = Project("X")

        with mock.patch('os.listdir') as mocked_listdir:
            mocked_listdir.return_value = ['alpha', 'beta', 'project.conf']
            project.load()
            self.assertEqual(len(project.groups), 2)

This does mock os.listdir successfully. But I can’t trick Python to treat mocked_listdir.return_value as consisting of files and directories.

How do I mock either os.listdir or os.path.isdir, in the same test, such that the test will see alpha and beta as directories and project.conf as a file?

Asked By: Ishan De Silva

||

Source

Answer 1

It will depend, of course, on exactly which os functions you use, but it looks like mock.patch.multiple on os is just what you need. (Note that you may not need to patch path; many of its functions are lexical-only and do not care about the actual filesytem.)

Answered By: Davis Herring

Answer 2

I managed to achieve the desired behavior by passing an iterable to the side_effect attribute of the mocked isdir object.

import unittest
from unittest import mock
import Project

class TestRepo(unittest.TestCase):

  def test_load_project(self):
      project = Project("X")

      with mock.patch('os.listdir') as mocked_listdir:
        with mock.patch('os.path.isdir') as mocked_isdir:
          mocked_listdir.return_value = ['alpha', 'beta', 'project.conf']
          mocked_isdir.side_effect = [True, True, False]
          project.load()
          self.assertEqual(len(project.groups), 2)

The key is the mocked_isdir.side_effect = [True, True, False] line. The boolean values in the iterable should match the order of directory and file entries passed to the mocked_listdir.return_value attribute.

Answered By: Ishan De Silva

Answer 3

You could use pyfakefs (source, docs), a very handy lib to test operations on a fake file system.

If you use pytest, it has a plugin, all file system functions already get patched, you just need to use its fs fixture:

import os

def test_foo(fs):
    fs.CreateFile('/home/x/alpha/1')
    fs.CreateFile('/home/x/beta/2')
    fs.CreateFile('/home/x/p.conf')    
    assert os.listdir('/home/x/') == ['alpha', 'beta', 'p.conf']

or if you prefer unittest:

import os
import unittest

from pyfakefs import fake_filesystem_unittest
   
class TestRepo(fake_filesystem_unittest.TestCase):

    def setUp(self):
        self.setUpPyfakefs()

    def test_foo(self):
        os.makedirs('/home/x/alpha')
        os.makedirs('/home/x/beta')
        with open('/home/x/p.conf', 'w') as f:
            f.write('foo')
        self.assertEqual(os.listdir('/home/x/'), ['alpha', 'beta', 'p.conf'])


if __name__ == "__main__":
    unittest.main()

Answered By: georgexsh

How to mock os.listdir to pretend files and directories in Python?

Question:

Answers: