Remove empty dicts in nested dictionary with recursive function

Question

I’m trying to remove non-values from a nested dictionary. My first effort works fine, but unfortunately keys pointing to now empty dicts still persist.

So if i do:

pass1 = stripper(my_dict)
return stripper(pass1)

This works, but i’m thinking a more elegant nested solution might be possible?

def stripper(self, data):
    if isinstance(data, dict):
        d = ({k: stripper(v) for k, v in data.items()
             if v not in [u'', None]})
        if d:
            return d
    else:
        return data

Edit:

Failing example, dict below returns as {'foo': 'bar', 'bar': None}:

{
    'foo': 'bar',
    'bar': {
        'foo': None,
        'one': None
    }
}

Asked By: rix

||

Source

Answer 1

The dict comprehension is certainly concise but if you expand it out, the solution becomes more obvious:

def stripper(self, data):
    new_data = {}
    for k, v in data.items():
        if isinstance(v, dict):
            v = stripper(v)
        if not v in (u'', None, {}):
            new_data[k] = v
    return new_data

Answered By: Oliver Dain

Answer 2

To extend the accepted answer to include objects that might be an element of a list:

def strip_empties_from_list(data):
    new_data = []
    for v in data:
        if isinstance(v, dict):
            v = strip_empties_from_dict(v)
        elif isinstance(v, list):
            v = strip_empties_from_list(v)
        if v not in (None, str(), list(), dict(),):
            new_data.append(v)
    return new_data


def strip_empties_from_dict(data):
    new_data = {}
    for k, v in data.items():
        if isinstance(v, dict):
            v = strip_empties_from_dict(v)
        elif isinstance(v, list):
            v = strip_empties_from_list(v)
        if v not in (None, str(), list(), dict(),):
            new_data[k] = v
    return new_data

To use:

data = {
  'None': None,
  'empty_list': [],
  'empty_dict': {},
  'empty_string': '',
  'list_with_empties': ['', {}, [], None, {'more_empties': '', 'and_even_more': [''], 'one_thing_i_care_about': 'hi'}]
}
stripped_data = strip_empties_from_dict(data)
print(stripped_data)

Answered By: dnk8n

Answer 3

Answer of @Oliver is correct, but having some edge cases checks won’t hurt, here you go: (Was unable to edit Oliver’s answer as the queue was full)

    def dictionary_stripper(data):
        new_data = {}
        
        # Only iterate if the given dict is not None
        if data:
            for k, v in data.items():
                if isinstance(v, dict):
                    v = CampaignAdminUtils.dictionary_stripper(v)
                
                # ideally it should be not in, second you can also add a empty list if required
                if v not in ("", None, {}, []):
                    new_data[k] = v
            
            # Only if you want the root dict to be None if empty
            if new_data == {}:
                return None
            return new_data
        return None

Answered By: Sumit Badsara

Answer 4

I landed here with a need to drop None values yet preserve empty dicts and lists. Reused/modified @OliverDain answer, thanks for that. Hope this helps someone.

def drop_none_values(data):
    """
    Recursively remove None values in a dictionary or list;
    this includes list of dicts, etc. Do not drop empty
    dicts or lists.

    :param data: Object to process for None values.
        Returns a new dict if passed a dict;
        returns a new list if passed a list;
        returns the argument unchanged otherwise.
    """
    if isinstance(data, dict):
        new_data = {}
        for k, v in data.items():
            v = drop_none_values(v)
            if v is not None:
                new_data[k] = v
    elif isinstance(data, list):
        new_data = []
        for v in data:
            v = drop_none_values(v)
            if v is not None:
                new_data.append(v)
    else:
        new_data = data
    return new_data

Here’s my test case, that function above is in a file/module named util:

def test_drop_none_values():
    data = {
        'first': None,
        'second': None,
        'empty': [],
        'alist': ['string', None, {'foo': None, 'bar': 'baz'}]
    }
    assert len(data) == 4
    stripped = util.drop_none_values(data)
    assert len(stripped) == 2
    assert len(stripped['alist']) == 2
    assert len(stripped['alist'][1]) == 1

Answered By: chrisinmtown

Remove empty dicts in nested dictionary with recursive function

Question:

Answers: