How to merge two list of dictionaries based on a value

Question

I have two lists of dictionaries, lets say:

a = [{'id': 1, 'name': 'a'}]
b = [{'id': 1, 'city': 'b'}]

I want to have a list that merges every dictionary in both lists with the same ID. In this example i expect to have:

a = [{'id': 1, 'name': 'a', 'city': 'b'}]

Is there any cleaner way of doing it other than a for nested into the other?

Thanks

Asked By: Javi DR

||

Source

Answer 1

One way to do this is to make a dictionary, mapping the identifier that you want to use (id in this case) to a dictionary of merged results.

#!/usr/bin/python

import collections

def merge_on_key(list_of_dictionaries, key, result):
    for d in list_of_dictionaries:
        assert(key in d)
        result[d[key]].update(d)

a = [{'id': 1, 'name': 'a'}]
b = [{'id': 1, 'city': 'b'}, {'id': 2, 'color': 'blue'}]

print 'a', a
print 'b', b

c = collections.defaultdict(lambda: {})
merge_on_key(a, 'id', c)
merge_on_key(b, 'id', c)

print 'merged results in dictionary with id 1', c[1]

That returns:

merged results in dictionary with id 1 {'city': 'b', 'id': 1, 'name': 'a'}

Answered By: James Thompson

Answer 2

You can use map, lambda function in conjunction with update method for dictionaries, like this:

a = [{'id': 1, 'name': 'a'}, {'id': 2, 'name': 'a'}, {'id': 3, 'name': 'k'}]
b = [{'id': 1, 'city': 'b'}, {'id': 2, 'city': 'c'},  {'id': 4, 'city': 'cm'}]
a.extend(list(map(lambda x,y: y if x.get('id') != y.get('id') else x.update(y), a, b)))
a = list(filter(None, a))

a will now become a list containing dictionaries of merged values like this:

[{'id': 1, 'name': 'a', 'city': 'b'},
 {'id': 2, 'name': 'a', 'city': 'c'},
 {'id': 3, 'name': 'k'},
 {'id': 4, 'city': 'cm'}]

Answered By: Anup Tiwari

Answer 3

You can keep track of the ids with another dict (or defaultdict to make things simpler). Then update the items in that dict as you iterate. In the end the dict’s values will have your list.

from collections import defaultdict
d = defaultdict(dict)

a = [{'id': 1, 'name': 'a'}, {'id': 3, 'name': 'a'}]
b = [{'id': 1, 'city': 'b'}, {'id': 2, 'city': 'c'}, {'id': 3, 'city': 'd'}]

for item in a + b:
    d[item['id']].update(item)
list(d.values())

# [{'id': 1, 'name': 'a', 'city': 'b'},
#  {'id': 3, 'name': 'a', 'city': 'd'},
#  {'id': 2, 'city': 'c'}]

Note this will overwrite duplicate values other than id — so if you have two with id: 1 and two different cities, you will only get the last city.

Answered By: Mark

Answer 4

from collections import defaultdict
from operator import itemgetter
l1 =[{'id': 1, 'City': 'Calcutta'}, {'id': 3, 'Country': 'Germany'}]
l2 = [{'id': 1, 'Country': 'India'}, {'id': 2, 'City': 'Delhi'}, {'id': 3, 'City': 'Berlin'}]

def merge1(l1,l2):
    d = defaultdict(dict)
    for l in (l1, l2):
        for innerdict1 in l:
            d[innerdict1['id']].update(innerdict1)

    l4 = sorted(d.values(), key=itemgetter("id"))
    l4p = print(l4)
    return l4p
merge1(l1, l2)

"""
[{'id': 1, 'City': 'Delhi', 'Country': 'India'}, {'id': 2, 'City': 'Calcutta'}, {'id': 3, 'Country': 'Germany', 'City': 'Berlin'}]

"""

Answered By: Soudipta Dutta

How to merge two list of dictionaries based on a value

Question:

Answers: