Sum list of dicts of lists

Question

I have a list of dicts, every value in a dict is a four-element list:

my_dict=[
    {
      'prop1': [1, 2, 3, 4],
      'prop2': [1, 1, 0, 0]
    },
    {
      'prop1': [2, 3, 3, 1],
      'prop3': [1, 1, 0, 0]
    }
]

Is it possible to sum it up without writing explicit iteration?

I want to get:

my_dict_sum={
      'prop1': [3, 5, 6, 5],
      'prop2': [1, 1, 0, 0],
      'prop3': [1, 1, 0, 0]
}

UPD: something like this works, but I wonder how to use map or zip or functools to do the same without writing two levels of iteration:

my_dict_sum = {}
for val in my_dict:
  for key,counts in val.items():
    if key in my_dict_sum :
        sum_dict[key] = list(map(lambda x,y: x+y, my_dict_sum[key], counts))
    else:
        my_dict_sum[key] = counts

Asked By: rfg

||

Source

Answer 1

EDIT:

Here is a horrific one-liner with no explicit loops. I’m not sure why you’d ever want to do this. You SHOULDN’T do this. Assuming you don’t count comprehension as explicit loops despite the presence of the word "for".

from functools import reduce

output = {k: reduce(
    lambda a, b:
        map(lambda d, e:
            d+e, a, b[k] if k in b else [0,0,0,0]
        ),
        dicts, 
        [0, 0, 0, 0]
    )
    for k in reduce(
        lambda s, b:
            s | set(b.keys()),
        dicts, set()
    )
}

Here’s a simpler version with explicit loops, that I feel is a lot clearer than your code so maybe that’s sufficient.

from collections import defaultdict

dicts = [
    {
      'prop1': [1, 2, 3, 4],
      'prop2': [1, 1, 0, 0]
    },
    {
      'prop1': [2, 3, 3, 1],
      'prop3': [1, 1, 0, 0]
    }
]

output = defaultdict(lambda: [0, 0, 0, 0])

for d in dicts:
    for k, l in d.items():
        for i, v in enumerate(l):
            output[k][i] += v

Note that it will only work if all the lists have exactly four items. For handling the case where you don’t know how many items are in the lists, you can do this slightly more complex version:

output = defaultdict(list)

for d in dicts:
    for k, l in d.items():
        for i, v in enumerate(l):
            if i >= len(output[k]):
                output[k].append(v)
            else:
                output[k][i] += v

You could do this as a one-liner using reduce and map and a bunch of lambdas, but it would be an incomprehensible mess. If you’re concerned about a bunch of loops inline in your code, just put it in a function and call the function.

Answered By: Simon Lundberg

Answer 2

Update:
probably the better way is to define your own dictionary using the builtin UserDict Class, like this:

from collections import UserDict
class mydict(UserDict):  
    def __add__(self, other):
        for k, v in other.items():
            if k in self.data:
                self.data[k] =  list(map(sum, zip_longest(self.data[k], other[k], fillvalue=0)))
            else:
                self.data[k] = other.data[k]
        return self
    def __radd__(self, other):
        return self if other == 0 else self.__add__(other)

and now the input list to be converted to new mydict class:(this step could be skipped if data originally created as mydict instead of dict)

new_dict = [mydict(v) for v in my_dict]

now only thing you need to do is the sum:

sum(new_dict)

results:
{'prop1': [3, 5, 6, 5], 'prop2': [1, 1, 0, 0], 'prop3': [1, 1, 0, 0]}

previous solution:

you could do somthing like this:

from itertools import zip_longest
from collections import defaultdict
out = defaultdict(list)
for line in my_dict:
     for key, val in line.items():
        out[key] = list(map(sum, zip_longest(val, out[key], fillvalue=0)))

and results:

defaultdict(list,
            {'prop1': [3, 5, 6, 5],
             'prop2': [1, 1, 0, 0],
             'prop3': [1, 1, 0, 0]})

Answered By: amirhm

Answer 3

Considering that it’s not about finding the most explicit, or more pythonic, or most performant approach, but you just curious whether it hypothetically can be achieved in any other alternative way, here a short trick without any mentioning of for loop:

from itertools import chain

d = {}
items_chain = chain.from_iterable(map(lambda d: list(d.items()), my_dicts))
list(map(lambda t, d=d: d.update([t]) if t[0] not in d else 
    d.update([(t[0], list(map(sum, zip(d[t[0]], t[1]))))]), items_chain))
print(d)

{'prop1': [3, 5, 6, 5], 'prop2': [1, 1, 0, 0], 'prop3': [1, 1, 0, 0]}

Answered By: RomanPerekhrest

Sum list of dicts of lists

Question:

Answers: