Django serializer test post of file with user information

Question

I try to test a file upload like this:

@deconstructible
class FileGenerator:

    @staticmethod
    def generate_text_file(file_ending='txt'):
        file_content = b'some test string'
        file = io.BytesIO(file_content)
        file.name = f'test.{file_ending}'
        file.seek(0)
        return file

def test_this(self, api_client, login_as):
    user = login_as('quality-controller')
    url = reverse('test-list')
    organization = Organization(name="test")
    organization.save()

    data = {
        "organization": organization.id,
        "import_file": FileGenerator.generate_text_file('txt'),
        "user": {
            "id": user.id,
            "username": user.username,
        }
    }

    response = api_client.post(url, data, format='json')

But I receive the following error message:

b'{"import_file": ["The submitted data was not a file. Check the
encoding type on the form."]}’

I also tried to use: format='multipart' but then I receive the following error:

AssertionError: Test data contained a dictionary value for key ‘user’,
but multipart uploads do not support nested data. You may want to
consider using format=’json’ in this test case.

How can I solve this?

Asked By: Andreas

||

Source

Answer 1

This is how I deal with this issue:

Simplest: flatten the form

Suck it up and just remove the issue by making your serializer to use user_id and user_username and fix it up on the server side in the serializer’s validate(self, attrs) method. A bit ugly/hacky but it works just fine and can be documented.

def validate(self, attrs):
    attrs["user"] = {
        "id": attrs.pop("user_id"), 
        "name": attrs.pop("user_username")
    }
    return attrs

Nicest if you dont mind the size: B64 Fields

You can base64 encode the file field and pass it in the json. Then to decode it on the server side you would write (or search for) a simple Base64FileField() for DRF.

class UploadedBase64ImageSerializer(serializers.Serializer):
    file = Base64ImageField(required=False)
    created = serializers.DateTimeField()

Alternative – Flatten the form data

You can’t pass nested data, but you can flatten the nested dicts and pass that to a DRF service. Serializers actually can understand nested data if the field names are correct.

I don’t know if this field name format is standardized, but this is what worked for me after experimentation. I only use it for service->service communication TO drf, so you would have to clone it into JS, but you can use the python in unit tests. Let me know if it works for you.

def flatten_dict_for_formdata(input_dict, array_separator="[{i}]"):
    """
    Recursively flattens nested dict()s into a single level suitable
    for passing to a library that makes multipart/form-data posts.
    """

    def __flatten(value, prefix, result_dict, previous=None):
        if isinstance(value, dict):
            # If we just processed a dict, then separate with a "."
            # Don't do this if it is an object inside an array.  
            # In that case the [:id] _is_ the separator, adding 
            # a "." like list[1].name will break but list[x]name 
            # is correct (at least for DRF/django decoding)
            if previous == "dict":
                prefix += "."

            for key, v in value.items():
                __flatten(
                    value=v,
                    prefix=prefix + key,
                    result_dict=result_dict,
                    previous="dict"
                )
        elif isinstance(value, list) or isinstance(value, tuple):
            for i, v in enumerate(value):
                __flatten(
                    value=v,
                    prefix=prefix + array_separator.format(i=i),  # e.g. name[1]
                    result_dict=result_dict,
                    previous="array"
                )
        else:
            result_dict[prefix] = value

        # return her to simplify the caller's life.  ignored during recursion
        return result_dict

    return __flatten(input_dict, '', OrderedDict(), None)

# flatten_dict_for_formdata({...}):
{                                   # output field name
    "file": SimpleUploadFile(...),  # file
    "user": {                       
        "id": 1,                    # user.id
        "name": "foghorn",          # user.name        
        "jobs": [                           
            "driver",               # user.jobs[0]
            "captain",              # user.jobs[1]
            "pilot"                 # user.jobs[1]
        ]
    },
    "objects": [
        {
            "type": "shoe",         # objects[0]type
            "size": "44"            # objects[0]size
        },
    ]
}

Answered By: Andrew

Django serializer test post of file with user information

Question:

Answers:

Simplest: flatten the form

Nicest if you dont mind the size: B64 Fields

Alternative – Flatten the form data