How to get the difference between two dictionaries in Python?

You were right to look at using a set, we just need to dig in a little deeper to get your method to work.

First, the example code:

test_1 = {"foo": "bar", "FOO": "BAR"}
test_2 = {"foo": "bar", "f00": "b@r"}

We can see right now that both dictionaries contain a similar key/value pair:

{"foo": "bar", ...}

Each dictionary also contains a completely different key value pair. But how do we detect the difference? Dictionaries don’t support that. Instead, you’ll want to use a set.

Here is how to turn each dictionary into a set we can use:

set_1 = set(test_1.items())
set_2 = set(test_2.items())

This returns a set containing a series of tuples. Each tuple represents one key/value pair from your dictionary.

Now, to find the difference between set_1 and set_2:

print set_1 - set_2
>>> {('FOO', 'BAR')}

Want a dictionary back? Easy, just:

dict(set_1 - set_2)
>>> {'FOO': 'BAR'}

I think it’s better to use the symmetric difference operation of sets to do that Here is the link to the doc.

>>> dict1 = {1:'donkey', 2:'chicken', 3:'dog'}
>>> dict2 = {1:'donkey', 2:'chimpansee', 4:'chicken'}
>>> set1 = set(dict1.items())
>>> set2 = set(dict2.items())
>>> set1 ^ set2
{(2, 'chimpansee'), (4, 'chicken'), (2, 'chicken'), (3, 'dog')}

It is symmetric because:

>>> set2 ^ set1
{(2, 'chimpansee'), (4, 'chicken'), (2, 'chicken'), (3, 'dog')}

This is not the case when using the difference operator.

>>> set1 - set2
{(2, 'chicken'), (3, 'dog')}
>>> set2 - set1
{(2, 'chimpansee'), (4, 'chicken')}

However it may not be a good idea to convert the resulting set to a dictionary because you may lose information:

>>> dict(set1 ^ set2)
{2: 'chicken', 3: 'dog', 4: 'chicken'}

Another solution would be dictdiffer (https://github.com/inveniosoftware/dictdiffer).

import dictdiffer                                          

a_dict = {                                                 
  'a': 'foo',
  'b': 'bar',
  'd': 'barfoo'
}                                                          

b_dict = {                                                 
  'a': 'foo',                                              
  'b': 'BAR',
  'c': 'foobar'
}                                                          

for diff in list(dictdiffer.diff(a_dict, b_dict)):         
    print diff

A diff is a tuple with the type of change, the changed value, and the path to the entry.

('change', 'b', ('bar', 'BAR'))
('add', '', [('c', 'foobar')])
('remove', '', [('d', 'barfoo')])

This function gives you all the diffs (and what stayed the same) based on the dictionary keys only. It also highlights some nice Dict comprehension, Set operations and python 3.6 type annotations 🙂

from typing import Dict, Any, Tuple
def get_dict_diffs(a: Dict[str, Any], b: Dict[str, Any]) -> Tuple[Dict[str, Any], Dict[str, Any], Dict[str, Any], Dict[str, Any]]:

    added_to_b_dict: Dict[str, Any] = {k: b[k] for k in set(b) - set(a)}
    removed_from_a_dict: Dict[str, Any] = {k: a[k] for k in set(a) - set(b)}
    common_dict_a: Dict[str, Any] = {k: a[k] for k in set(a) & set(b)}
    common_dict_b: Dict[str, Any] = {k: b[k] for k in set(a) & set(b)}
    return added_to_b_dict, removed_from_a_dict, common_dict_a, common_dict_b

If you want to compare the dictionary values:

values_in_b_not_a_dict = {k : b[k] for k, _ in set(b.items()) - set(a.items())}

What about this? Not as pretty but explicit.

orig_dict = {'a' : 1, 'b' : 2}
new_dict = {'a' : 2, 'v' : 'hello', 'b' : 2}

updates = {}
for k2, v2 in new_dict.items():
    if k2 in orig_dict:    
        if v2 != orig_dict[k2]:
            updates.update({k2 : v2})
    else:
        updates.update({k2 : v2})

#test it
#value of 'a' was changed
#'v' is a completely new entry
assert all(k in updates for k in ['a', 'v'])

def flatten_it(d):
    if isinstance(d, list) or isinstance(d, tuple):
        return tuple([flatten_it(item) for item in d])
    elif isinstance(d, dict):
        return tuple([(flatten_it(k), flatten_it(v)) for k, v in sorted(d.items())])
    else:
        return d

dict1 = {'a': 1, 'b': 2, 'c': 3}
dict2 = {'a': 1, 'b': 1}

print set(flatten_it(dict1)) - set(flatten_it(dict2)) # set([('b', 2), ('c', 3)])
# or 
print set(flatten_it(dict2)) - set(flatten_it(dict1)) # set([('b', 1)])

A function using the symmetric difference set operator, as mentioned in other answers, which preserves the origins of the values:

def diff_dicts(a, b, missing=KeyError):
    """
    Find keys and values which differ from `a` to `b` as a dict.

    If a value differs from `a` to `b` then the value in the returned dict will
    be: `(a_value, b_value)`. If either is missing then the token from 
    `missing` will be used instead.

    :param a: The from dict
    :param b: The to dict
    :param missing: A token used to indicate the dict did not include this key
    :return: A dict of keys to tuples with the matching value from a and b
    """
    return {
        key: (a.get(key, missing), b.get(key, missing))
        for key in dict(
            set(a.items()) ^ set(b.items())
        ).keys()
    }

Example

print(diff_dicts({'a': 1, 'b': 1}, {'b': 2, 'c': 2}))

# {'c': (<class 'KeyError'>, 2), 'a': (1, <class 'KeyError'>), 'b': (1, 2)}

How this works

We use the symmetric difference set operator on the tuples generated from taking items. This generates a set of distinct (key, value) tuples from the two dicts.

We then make a new dict from that to collapse the keys together and iterate over these. These are the only keys that have changed from one dict to the next.

We then compose a new dict using these keys with a tuple of the values from each dict substituting in our missing token when the key isn’t present.

Old question, but thought I’d share my solution anyway. Pretty simple.

dicta_set = set(dicta.items()) # creates a set of tuples (k/v pairs)
dictb_set = set(dictb.items())
setdiff = dictb_set.difference(dicta_set) # any set method you want for comparisons
for k, v in setdiff: # unpack the tuples for processing
    print(f"k/v differences = {k}: {v}")

This code creates two sets of tuples representing the k/v pairs. It then uses a set method of your choosing to compare the tuples. Lastly, it unpacks the tuples (k/v pairs) for processing.

This will return a new dict (only changed data).

def get_difference(obj_1: dict, obj_2: dict) -> dict:
result = {}

for key in obj_1.keys():
    value = obj_1[key]

    if isinstance(value, dict):
        difference = get_difference(value, obj_2.get(key, {}))

        if difference:
            result[key] = difference

    elif value != obj_2.get(key):
        result[key] = obj_2.get(key, None)

return result

Not sure this is what the OP asked for, but this is what I was looking for when I came across this question – specifically, how to show key by key the difference between two dicts:

Pitfall: when one dict has a missing key, and the second has it with a None value, the function would assume they are similar

This is not optimized at all – suitable for small dicts

def diff_dicts(a, b, drop_similar=True):
    res = a.copy()

    for k in res:
        if k not in b:
            res[k] = (res[k], None)

    for k in b:
        if k in res:
            res[k] = (res[k], b[k])
        else:
            res[k] = (None, b[k])

    if drop_similar:
        res = {k:v for k,v in res.items() if v[0] != v[1]}

    return res


print(diff_dicts({'a': 1}, {}))
print(diff_dicts({'a': 1}, {'a': 2}))
print(diff_dicts({'a': 2}, {'a': 2}))
print(diff_dicts({'a': 2}, {'b': 2}))
print(diff_dicts({'a': 2}, {'a': 2, 'b': 1}))

Output:

{'a': (1, None)}
{'a': (1, 2)}
{}
{'a': (2, None), 'b': (None, 2)}
{'b': (None, 1)}

A solution is to use the unittest module:

from unittest import TestCase
TestCase().assertDictEqual(expected_dict, actual_dict)

Obtained from How can you test that two dictionaries are equal with pytest in python

This is my own version, from combining https://stackoverflow.com/a/67263119/919692 with https://stackoverflow.com/a/48544451/919692, and now I see it is quite similar to https://stackoverflow.com/a/47433207/919692:

def dict_diff(dict_a, dict_b, show_value_diff=True):
  result = {}
  result['added']   = {k: dict_b[k] for k in set(dict_b) - set(dict_a)}
  result['removed'] = {k: dict_a[k] for k in set(dict_a) - set(dict_b)}
  if show_value_diff:
    common_keys =  set(dict_a) & set(dict_b)
    result['value_diffs'] = {
      k:(dict_a[k], dict_b[k])
      for k in common_keys
      if dict_a[k] != dict_b[k]
    }
  return result

I would recommend using something already written by good developers. Like pytest. It has a deal with any data type, not only dicts. And, BTW, pytest is very good at testing.

from _pytest.assertion.util import _compare_eq_any

print('n'.join(_compare_eq_any({'a': 'b'}, {'aa': 'vv'}, verbose=3)))

Output is:

Left contains 1 more item:
{'a': 'b'}
Right contains 1 more item:
{'aa': 'vv'}
Full diff:
- {'aa': 'vv'}
?    -    ^^
+ {'a': 'b'}
?        ^

If you don’t like using private functions (started with _), just have a look at the source code and copy/paste the function to your code.

P.S.: Tested with pytest==6.2.4

You can use DeepDiff:

pip install deepdiff

Among other things, it lets you recursively calculate the difference of dictionaries, iterables, strings and other objects:

>>> from deepdiff import DeepDiff

>>> d1 = {1:1, 2:2, 3:3, "foo":4}
>>> d2 = {1:1, 2:4, 3:3, "bar":5, 6:6}
>>> DeepDiff(d1, d2)
{'dictionary_item_added': [root['bar'], root[6]],
 'dictionary_item_removed': [root['foo']],
 'values_changed': {'root[2]': {'new_value': 4, 'old_value': 2}}}

It lets you see what changed (even types), what was added and what was removed. It also lets you do many other things like ignoring duplicates and ignoring paths (defined by regex).

Here is a variation that lets you update dict1 values if you know the values in dict2 are right.

Consider:

dict1.update((k, dict2.get(k)) for k, v in dict1.items())

For one side comparison you can use dict comprehension:

dict1 = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
dict2 = {'a': OMG, 'b': 2, 'c': 3, 'd': 4}

data = {a:dict1[a] for a in dict1 if dict1[a] != dict2[a]}

output: {‘a’: 1}

a_dic={'a':1, 'b':2}
b_dic={'a':1, 'b':20}

sharedmLst = set(a_dic.items()).intersection(b_dic.items())
diff_from_b = set(a_dic.items()) - sharedmLst
diff_from_a = set(b_dic.items()) - sharedmLst

print("Among the items in a_dic, the item different from b_dic",diff_from_b)
print("Among the items in b_dic, the item different from a_dic",diff_from_a)

Result :
Among the items in a_dic, the item different from b_dic {('b', 2)}
Among the items in b_dic, the item different from a_dic {('b', 20)}

This solution works perfectly with unhashable dicts, which avoids this error:

TypeError: Unhashable type 'dict'.

Start with the top-ranked solution from @Roedy. We create a dictionary of lists, which are a good example of something that is non-hashable:

>>> dict1 = {1:['donkey'], 2:['chicken'], 3:['dog']}
>>> dict2 = {1:['donkey'], 2:['chimpansee'], 4:['chicken']}

Then we preprocess to make each value hashable using str(value):

>>> set1 = set([(key, str(value)) for key, value in dict1.items()])
>>> set2 = set([(key, str(value)) for key, value in dict2.items()])

Then we continue as per answer from @Reody:

>>> set1 ^ set2
{(3, "['dog']"), (4, "['chicken']"), (2, "['chimpansee']"), (2,"['chicken']")}