Compare dictionaries ignoring specific keys

Question

How can I test if two dictionaries are equal while taking some keys out of consideration. For example,

equal_dicts(
    {'foo':1, 'bar':2, 'x':55, 'y': 77 },
    {'foo':1, 'bar':2, 'x':66, 'z': 88 },
    ignore_keys=('x', 'y', 'z')
)

should return True.

UPD: I’m looking for an efficient, fast solution.

UPD2. I ended up with this code, which appears to be the fastest:

def equal_dicts_1(a, b, ignore_keys):
    ka = set(a).difference(ignore_keys)
    kb = set(b).difference(ignore_keys)
    return ka == kb and all(a[k] == b[k] for k in ka)

Timings: https://gist.github.com/2651872

Asked By: georg

||

Source

Answer 1

def equal_dicts(d1, d2, ignore_keys):
    d1_filtered = {k:v for k,v in d1.items() if k not in ignore_keys}
    d2_filtered = {k:v for k,v in d2.items() if k not in ignore_keys}
    return d1_filtered == d2_filtered

EDIT: This might be faster and more memory-efficient:

def equal_dicts(d1, d2, ignore_keys):
    ignored = set(ignore_keys)
    for k1, v1 in d1.iteritems():
        if k1 not in ignored and (k1 not in d2 or d2[k1] != v1):
            return False
    for k2, v2 in d2.iteritems():
        if k2 not in ignored and k2 not in d1:
            return False
    return True

Answered By: eumiro

Answer 2

Very very crudely, you could just delete any ignored keys and compare those dictionaries:

def equal_dicts(d1, d2, ignore_keys=()):
    d1_, d2_ = d1.copy(), d2.copy()
    for k in ignore_keys:
        try:
            del d1_[k]
        except KeyError: 
            pass
        try:
            del d2_[k]
        except KeyError: 
            pass

    return d1_ == d2_

(Note that we don’t need a deep copy here, we just need to avoid modifying d1 and d2.)

Answered By: huon

Answer 3

def compare_dict(d1, d2, ignore):
    for k in d1:
        if k in ignore:
            continue
        try:
            if d1[k] != d2[k]:
                return False
        except KeyError:
            return False
    return True

Comment edit: You can do something like compare_dict(d1, d2, ignore) and compare_dict(d2, d1, ignore) or duplicate the for

def compare_dict(d1, d2, ignore):
    ignore = set(ignore)
    for k in d1:
        if k in ignore:
            continue
        try:
            if d1[k] != d2[k]:
                return False
        except KeyError:
            return False

    for k in d2:
        if k in ignore:
            continue
        try:
            if d1[k] != d2[k]:
                return False
        except KeyError:
            return False
    return True

Whatever is faster and cleaner!
Update: cast set(ignore)

Answered By: StefanNch

Answer 4

Using dict comprehensions:

>>> {k: v for k,v in d1.items() if k not in ignore_keys} == 
... {k: v for k,v in d2.items() if k not in ignore_keys}

Use .viewitems() instead on Python 2.

Answered By: wim

Answer 5

Optimal solution for the case of ignoring only one key

return all(
    (x == y or (x[1] == y[1] == 'key to ignore')) for x, y in itertools.izip(
          d1.iteritems(), d2.iteritems()))

Answered By: Andre Yanpolsky

Answer 6

in case your dictionary contained lists or other dictionaries:

def equal_dicts(d1, d2, ignore_keys, equal):
    # print('got d1', d1)
    # print('got d2', d2)
    if isinstance(d1, str):
        if not isinstance(d2, str):
            return False
        return d1 == d2
    for k in d1:
        if k in ignore_keys:
            continue
        if not isinstance(d1[k], dict) and not isinstance(d1[k], list) and d2.get(k) != d1[k]:
            print(k)
            equal = False
        elif isinstance(d1[k], list):
            if not isinstance(d2.get(k), list):
                equal = False
            if len(d1[k]) != len(d2[k]):
                return False
            if len(d1[k]) > 0 and isinstance(d1[k][0], dict):
                if not isinstance(d2[k][0], dict):
                    return False
                d1_sorted = sorted(d1[k], key=lambda item: item.get('created'))
                d2_sorted = sorted(d2[k], key=lambda item: item.get('created'))
                equal = all(equal_dicts(x, y, ignore_keys, equal) for x, y in zip(d1_sorted, d2_sorted)) and equal
            else:
                equal = all(equal_dicts(x, y, ignore_keys, equal) for x, y in zip(d1[k], d2[k])) and equal
        elif isinstance(d1[k], dict):
            if not isinstance(d2.get(k), dict):
                equal = False
            print(k)
            equal = equal_dicts(d1[k], d2[k], ignore_keys, equal) and equal
    return equal

Answered By: Ahmad sibai

Answer 7

Here’s another variant:

set(ignore_keys).issuperset(k for (k, v) in d1.items() ^ d2.items())

Its virtues:

C speed identification of differences between the dicts
C speed check for membership in the set of ignored keys
Early-out if a single mismatch is found

Answered By: Raymond Hettinger

Answer 8

If you need this check when testing, you can use the ANY from the unittest.mock library.
Here is an example.

from unittest.mock import ANY
actual = {'userName':'bob', 'lastModified':'2012-01-01'}
expected = {'userName':'bob', 'lastModified': ANY}
assert actual == expected

Answered By: Alexander Rakhmaev

Compare dictionaries ignoring specific keys

Question:

Answers: