How do I merge dictionaries together in Python?
Question:
d3 = dict(d1, **d2)
I understand that this merges the dictionary. But, is it unique? What if d1 has the same key as d2 but different value? I would like d1 and d2 to be merged, but d1 has priority if there is duplicate key.
Answers:
You can use the .update() method if you don’t need the original d2 any more:
Update the dictionary with the key/value pairs from other, overwriting existing keys. Return None.
E.g.:
>>> d1 = {'a': 1, 'b': 2}
>>> d2 = {'b': 1, 'c': 3}
>>> d2.update(d1)
>>> d2
{'a': 1, 'c': 3, 'b': 2}
Update:
Of course you can copy the dictionary first in order to create a new merged one. This might or might not be necessary. In case you have compound objects (objects that contain other objects, like lists or class instances) in your dictionary, copy.deepcopy should also be considered.
In Python2,
d1={'a':1,'b':2}
d2={'a':10,'c':3}
d1 overrides d2:
dict(d2,**d1)
# {'a': 1, 'c': 3, 'b': 2}
d2 overrides d1:
dict(d1,**d2)
# {'a': 10, 'c': 3, 'b': 2}
This behavior is not just a fluke of implementation; it is guaranteed in the documentation:
If a key is specified both in the
positional argument and as a keyword
argument, the value associated with
the keyword is retained in the
dictionary.
If you want d1 to have priority in the conflicts, do:
d3 = d2.copy()
d3.update(d1)
Otherwise, reverse d2 and d1.
My solution is to define a merge function. It’s not sophisticated and just cost one line. Here’s the code in Python 3.
from functools import reduce
from operator import or_
def merge(*dicts):
return { k: reduce(lambda d, x: x.get(k, d), dicts, None) for k in reduce(or_, map(lambda x: x.keys(), dicts), set()) }
Tests
>>> d = {0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
>>> d_letters = {0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e', 5: 'f', 6: 'g', 7: 'h', 8: 'i', 9: 'j', 10: 'k', 11: 'l', 12: 'm', 13: 'n', 14: 'o', 15: 'p', 16: 'q', 17: 'r', 18: 's', 19: 't', 20: 'u', 21: 'v', 22: 'w', 23: 'x', 24: 'y', 25: 'z', 26: 'A', 27: 'B', 28: 'C', 29: 'D', 30: 'E', 31: 'F', 32: 'G', 33: 'H', 34: 'I', 35: 'J', 36: 'K', 37: 'L', 38: 'M', 39: 'N', 40: 'O', 41: 'P', 42: 'Q', 43: 'R', 44: 'S', 45: 'T', 46: 'U', 47: 'V', 48: 'W', 49: 'X', 50: 'Y', 51: 'Z'}
>>> merge(d, d_letters)
{0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e', 5: 'f', 6: 'g', 7: 'h', 8: 'i', 9: 'j', 10: 'k', 11: 'l', 12: 'm', 13: 'n', 14: 'o', 15: 'p', 16: 'q', 17: 'r', 18: 's', 19: 't', 20: 'u', 21: 'v', 22: 'w', 23: 'x', 24: 'y', 25: 'z', 26: 'A', 27: 'B', 28: 'C', 29: 'D', 30: 'E', 31: 'F', 32: 'G', 33: 'H', 34: 'I', 35: 'J', 36: 'K', 37: 'L', 38: 'M', 39: 'N', 40: 'O', 41: 'P', 42: 'Q', 43: 'R', 44: 'S', 45: 'T', 46: 'U', 47: 'V', 48: 'W', 49: 'X', 50: 'Y', 51: 'Z'}
>>> merge(d_letters, d)
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 'f', 6: 'g', 7: 'h', 8: 'i', 9: 'j', 10: 'k', 11: 'l', 12: 'm', 13: 'n', 14: 'o', 15: 'p', 16: 'q', 17: 'r', 18: 's', 19: 't', 20: 'u', 21: 'v', 22: 'w', 23: 'x', 24: 'y', 25: 'z', 26: 'A', 27: 'B', 28: 'C', 29: 'D', 30: 'E', 31: 'F', 32: 'G', 33: 'H', 34: 'I', 35: 'J', 36: 'K', 37: 'L', 38: 'M', 39: 'N', 40: 'O', 41: 'P', 42: 'Q', 43: 'R', 44: 'S', 45: 'T', 46: 'U', 47: 'V', 48: 'W', 49: 'X', 50: 'Y', 51: 'Z'}
>>> merge(d)
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
>>> merge(d_letters)
{0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e', 5: 'f', 6: 'g', 7: 'h', 8: 'i', 9: 'j', 10: 'k', 11: 'l', 12: 'm', 13: 'n', 14: 'o', 15: 'p', 16: 'q', 17: 'r', 18: 's', 19: 't', 20: 'u', 21: 'v', 22: 'w', 23: 'x', 24: 'y', 25: 'z', 26: 'A', 27: 'B', 28: 'C', 29: 'D', 30: 'E', 31: 'F', 32: 'G', 33: 'H', 34: 'I', 35: 'J', 36: 'K', 37: 'L', 38: 'M', 39: 'N', 40: 'O', 41: 'P', 42: 'Q', 43: 'R', 44: 'S', 45: 'T', 46: 'U', 47: 'V', 48: 'W', 49: 'X', 50: 'Y', 51: 'Z'}
>>> merge()
{}
It works for arbitrary number of dictionary arguments. Were there any duplicate keys in those dictionary, the key from the rightmost dictionary in the argument list wins.
Trey Hunner has a nice blog post outlining several options for merging multiple dictionaries, including (for python3.3+) ChainMap and dictionary unpacking.
I believe that, as stated above, using d2.update(d1) is the best approach and that you can also copy d2 first if you still need it.
Although, I want to point out that dict(d1, **d2) is actually a bad way to merge dictionnaries in general since keyword arguments need to be strings, thus it will fail if you have a dict such as:
{
1: 'foo',
2: 'bar'
}
Starting in Python 3.9, the operator | creates a new dictionary with the merged keys and values from two dictionaries:
# d1 = { 'a': 1, 'b': 2 }
# d2 = { 'b': 1, 'c': 3 }
d3 = d2 | d1
# d3: {'b': 2, 'c': 3, 'a': 1}
This:
Creates a new dictionary d3 with the merged keys and values of d2 and d1. The values of d1 take priority when d2 and d1 share keys.
Also note the |= operator which modifies d2 by merging d1 in, with priority on d1 values:
# d1 = { 'a': 1, 'b': 2 }
# d2 = { 'b': 1, 'c': 3 }
d2 |= d1
# d2: {'b': 2, 'c': 3, 'a': 1}
d3 = dict(d1, **d2)
I understand that this merges the dictionary. But, is it unique? What if d1 has the same key as d2 but different value? I would like d1 and d2 to be merged, but d1 has priority if there is duplicate key.
You can use the .update() method if you don’t need the original d2 any more:
Update the dictionary with the key/value pairs from other, overwriting existing keys. Return
None.
E.g.:
>>> d1 = {'a': 1, 'b': 2}
>>> d2 = {'b': 1, 'c': 3}
>>> d2.update(d1)
>>> d2
{'a': 1, 'c': 3, 'b': 2}
Update:
Of course you can copy the dictionary first in order to create a new merged one. This might or might not be necessary. In case you have compound objects (objects that contain other objects, like lists or class instances) in your dictionary, copy.deepcopy should also be considered.
In Python2,
d1={'a':1,'b':2}
d2={'a':10,'c':3}
d1 overrides d2:
dict(d2,**d1)
# {'a': 1, 'c': 3, 'b': 2}
d2 overrides d1:
dict(d1,**d2)
# {'a': 10, 'c': 3, 'b': 2}
This behavior is not just a fluke of implementation; it is guaranteed in the documentation:
If a key is specified both in the
positional argument and as a keyword
argument, the value associated with
the keyword is retained in the
dictionary.
If you want d1 to have priority in the conflicts, do:
d3 = d2.copy()
d3.update(d1)
Otherwise, reverse d2 and d1.
My solution is to define a merge function. It’s not sophisticated and just cost one line. Here’s the code in Python 3.
from functools import reduce
from operator import or_
def merge(*dicts):
return { k: reduce(lambda d, x: x.get(k, d), dicts, None) for k in reduce(or_, map(lambda x: x.keys(), dicts), set()) }
Tests
>>> d = {0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
>>> d_letters = {0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e', 5: 'f', 6: 'g', 7: 'h', 8: 'i', 9: 'j', 10: 'k', 11: 'l', 12: 'm', 13: 'n', 14: 'o', 15: 'p', 16: 'q', 17: 'r', 18: 's', 19: 't', 20: 'u', 21: 'v', 22: 'w', 23: 'x', 24: 'y', 25: 'z', 26: 'A', 27: 'B', 28: 'C', 29: 'D', 30: 'E', 31: 'F', 32: 'G', 33: 'H', 34: 'I', 35: 'J', 36: 'K', 37: 'L', 38: 'M', 39: 'N', 40: 'O', 41: 'P', 42: 'Q', 43: 'R', 44: 'S', 45: 'T', 46: 'U', 47: 'V', 48: 'W', 49: 'X', 50: 'Y', 51: 'Z'}
>>> merge(d, d_letters)
{0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e', 5: 'f', 6: 'g', 7: 'h', 8: 'i', 9: 'j', 10: 'k', 11: 'l', 12: 'm', 13: 'n', 14: 'o', 15: 'p', 16: 'q', 17: 'r', 18: 's', 19: 't', 20: 'u', 21: 'v', 22: 'w', 23: 'x', 24: 'y', 25: 'z', 26: 'A', 27: 'B', 28: 'C', 29: 'D', 30: 'E', 31: 'F', 32: 'G', 33: 'H', 34: 'I', 35: 'J', 36: 'K', 37: 'L', 38: 'M', 39: 'N', 40: 'O', 41: 'P', 42: 'Q', 43: 'R', 44: 'S', 45: 'T', 46: 'U', 47: 'V', 48: 'W', 49: 'X', 50: 'Y', 51: 'Z'}
>>> merge(d_letters, d)
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 'f', 6: 'g', 7: 'h', 8: 'i', 9: 'j', 10: 'k', 11: 'l', 12: 'm', 13: 'n', 14: 'o', 15: 'p', 16: 'q', 17: 'r', 18: 's', 19: 't', 20: 'u', 21: 'v', 22: 'w', 23: 'x', 24: 'y', 25: 'z', 26: 'A', 27: 'B', 28: 'C', 29: 'D', 30: 'E', 31: 'F', 32: 'G', 33: 'H', 34: 'I', 35: 'J', 36: 'K', 37: 'L', 38: 'M', 39: 'N', 40: 'O', 41: 'P', 42: 'Q', 43: 'R', 44: 'S', 45: 'T', 46: 'U', 47: 'V', 48: 'W', 49: 'X', 50: 'Y', 51: 'Z'}
>>> merge(d)
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
>>> merge(d_letters)
{0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e', 5: 'f', 6: 'g', 7: 'h', 8: 'i', 9: 'j', 10: 'k', 11: 'l', 12: 'm', 13: 'n', 14: 'o', 15: 'p', 16: 'q', 17: 'r', 18: 's', 19: 't', 20: 'u', 21: 'v', 22: 'w', 23: 'x', 24: 'y', 25: 'z', 26: 'A', 27: 'B', 28: 'C', 29: 'D', 30: 'E', 31: 'F', 32: 'G', 33: 'H', 34: 'I', 35: 'J', 36: 'K', 37: 'L', 38: 'M', 39: 'N', 40: 'O', 41: 'P', 42: 'Q', 43: 'R', 44: 'S', 45: 'T', 46: 'U', 47: 'V', 48: 'W', 49: 'X', 50: 'Y', 51: 'Z'}
>>> merge()
{}
It works for arbitrary number of dictionary arguments. Were there any duplicate keys in those dictionary, the key from the rightmost dictionary in the argument list wins.
Trey Hunner has a nice blog post outlining several options for merging multiple dictionaries, including (for python3.3+) ChainMap and dictionary unpacking.
I believe that, as stated above, using d2.update(d1) is the best approach and that you can also copy d2 first if you still need it.
Although, I want to point out that dict(d1, **d2) is actually a bad way to merge dictionnaries in general since keyword arguments need to be strings, thus it will fail if you have a dict such as:
{
1: 'foo',
2: 'bar'
}
Starting in Python 3.9, the operator | creates a new dictionary with the merged keys and values from two dictionaries:
# d1 = { 'a': 1, 'b': 2 }
# d2 = { 'b': 1, 'c': 3 }
d3 = d2 | d1
# d3: {'b': 2, 'c': 3, 'a': 1}
This:
Creates a new dictionary d3 with the merged keys and values of d2 and d1. The values of d1 take priority when d2 and d1 share keys.
Also note the |= operator which modifies d2 by merging d1 in, with priority on d1 values:
# d1 = { 'a': 1, 'b': 2 }
# d2 = { 'b': 1, 'c': 3 }
d2 |= d1
# d2: {'b': 2, 'c': 3, 'a': 1}