What is the clean way to unittest FileField in django?

Question:

I have a model with a FileField. I want to unittest it. django test framework has great ways to manage database and emails. Is there something similar for FileFields?

How can I make sure that the unittests are not going to pollute the real application?

Thanks in advance

PS: My question is almost a duplicate of Django test FileField using test fixtures but it doesn’t have an accepted answer. Just want to re-ask if something new on this topic.

Asked By: luc

||
Source

Answers:

I normally test filefields in models using doctest

>>> from django.core.files import File
>>> s = SimpleModel()
>>> s.audio_file = File(open("media/testfiles/testaudio.wav"))
>>> s.save()
>>> ...
>>> s.delete()

If I need to I also test file uploads with test clients.

As for fixtures, I simply copy the files i need in a test folder, after modifying the paths in the fixture.

e.g.

In a fixture containing models with filefiels pointing to a directory named “audio”, you replace “audio”: “audio/audio.wav” with “audio”: “audio/test/audio.wav” .
Now all you have to do is copy the test folder, with the necessary files, in “audio” in the test setUp and then delete it in tearDown.

Not the cleanest way ever i think, but that’s what i do.

Answered By: Mirko Rossini

There are several ways you could tackle this but they’re all ugly since unit tests are supposed to be isolated but files are all about durable changes.

My unit tests don’t run on a system with production data so it’s been easy to simply reset the upload directory after each run with something like git reset --hard. This approach is in some ways the best simply because it involves no code changes and is guaranteed to work as long as you start with good test data.

If you don’t actually need to do anything with that file after testing your model’s save method, I’d recommend using python’s excellent Mock library to completely fake the File instance (i.e. something like mock_file = Mock(spec=django.core.files.File); mock_file.read.return_value = "fake file contents") so you can completely avoid changes to your file handling logic. The Mock library has a couple of ways to globally patch Django’s File class within a test method which is about as easy as this will get.

If you need to have a real file (i.e. for serving as part of a test, processing with an external script, etc.) you can use something similar to Mirko’s example and create a File object after making sure it’ll be stored somewhere appropriate – here are three ways to do that:

  • Have your test settings.MEDIA_ROOT point to a temporary directory (see the Python tempfile module’s mkdtemp function). This works fine as long as you have something like a separate STATIC_ROOT which you use for the media files which are part of your source code.
  • Use a custom storage manager
  • Set the file path manually on each File instance or have a custom upload_to function to point somewhere which your test setup/teardown process purges such as a test subdirectory under MEDIA_ROOT.

Edit: mock object library is new in python version 3.3. For older python versions check Michael Foord’s version

Answered By: Chris Adams

If you just want to create an object that requires FileField and don’t want to use this field then you can just pass any (existing or not) relative path like this:

example_object = models.ExampleModel({'file': "foo.bar"})
example_object.save()

Then it’s ready for use.

Answered By: Ctrl-C

Django provides a great way to do this – use a SimpleUploadedFile or a TemporaryUploadedFile. SimpleUploadedFile is generally the simpler option if all you need to store is some sentinel data:

from django.core.files.uploadedfile import SimpleUploadedFile

my_model.file_field = SimpleUploadedFile(
    "best_file_eva.txt",
    b"these are the file contents!"   # note the b in front of the string [bytes]
)

It’s one of django’s magical features-that-don’t-show-up-in-the-docs :). However it is referred to here and implemented here.

Limitations

Note that you can only put bytes in a SimpleUploadedFile since it’s implemented using BytesIO behind the scenes. If you need more realistic, file-like behavior you can use TemporaryUploadedFile.

For Python 2

If you’re stuck on python 2, skip the b prefix in the content:

my_model.file_field = SimpleUploadedFile(
    "best_file_eva.txt",
    "these are the file contents!" # no b
)
Answered By: waterproof

I guess that easiest way is to use ContentFile class:

file = ContentFile('text', 'name')
my_model = MyModel()
my_model.file = file
my_model.save()
Answered By: Nikolay Pavlin

There is some way to mock storage.

After reading django source code, the below is my implementation to avoid uploading file to s3 regardless of using default storage or assigning the storage in filefield or imagefield.

from unittest.mock import patch
from django.db.models.fields.files import FileField, FieldFile, ImageField, ImageFieldFile
from django.core.files.storage import FileSystemStorage
from rest_framework.test import APITestCase


class CustomFileSystemStorage(FileSystemStorage):
    def url(self, *args, **kwargs):
        return self.path(*args, **kwargs)


class CustomFieldFile(FieldFile):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.storage = CustomFileSystemStorage()
        
        
class CustomImageFieldFile(ImageFieldFile):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.storage = CustomFileSystemStorage()


class PatchMeta(type):
    """A metaclass to patch all inherited classes."""

    def __new__(meta, name, bases, attrs):
        cls = type.__new__(meta, name, bases, attrs)
        cls = patch.object(ImageField, 'attr_class', CustomImageFieldFile)(cls)
        cls = patch.object(FileField, 'attr_class', CustomFieldFile)(cls)
        return cls


class CustomBaseTestCase(APITestCase, metaclass=PatchMeta):
    """You can inherit this class to do any testcase for mocking storage"""
    pass

With inheriting CustomBaseTestCase class, you can change your storage to avoid uploading files to any server

Answered By: Arthur Hung