For a Python dictionary, does iterkeys offer any advantages over viewkeys?
Question:
In Python 2.7, dictionaries have both an iterkeys method and a viewkeys method (and similar pairs for values and items), giving two different ways to lazily iterate over the keys of the dictionary. The viewkeys method provides the principal feature of iterkeys, with iter(d.viewkeys()) effectively equivalent to d.iterkeys(). Additionally, objects returned viewkeys have convenient set-like features. There thus are strong reasons to favor viewkeys over iterkeys.
What about the other direction? Apart from compatibility with earlier versions of Python, are there any ways in which iterkeys would be preferable to viewkeys? Would anything be lost by just always using viewkeys?
Answers:
A dictionary view updates as the dictionary does, while an iterator does not necessarily do this.
This means if you work with the view, change the dictionary, then work with the view again, the view will have changed to reflect the dictionary’s new state.
They provide a dynamic view on the dictionary’s entries, which means that when the dictionary changes, the view reflects these changes.
Source
Example:
>>> test = {1: 2, 3: 4}
>>> a = test.iterkeys()
>>> b = test.viewkeys()
>>> del test[1]
>>> test[5] = 6
>>> list(a)
[3, 5]
>>> b
dict_keys([3, 5])
When changes are made to the size, an exception will be thrown:
>>> test = {1: 2, 3: 4}
>>> a = test.iterkeys()
>>> b = test.viewkeys()
>>> test[5] = 6
>>> list(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
RuntimeError: dictionary changed size during iteration
>>> b
dict_keys([1, 3, 5])
It’s also worth noting you can only iterate over a keyiterator once:
>>> test = {1: 2, 3: 4}
>>> a = test.iterkeys()
>>> list(a)
[1, 3]
>>> list(a)
[]
>>> b = test.viewkeys()
>>> b
dict_keys([1, 3])
>>> b
dict_keys([1, 3])
No, there’s no advantage to iterkeys over viewkeys, in the same way that there’s no advantage to keys over either of them. iterkeys is only around for back compatibility. Indeed, in Python 3, viewkeys is the only behaviour that still exists, and it has been renamed to keys – the viewkeys method is actually a backport of the Python 3 behaviour.
As the name (and documentation) indicate, the viewkeys(), viewvalues() and viewitems() methods return a view of the current elements in the dictionary meaning that if the dictionary changes, so does the view; views are lazy. In the general case keys views are set-like, and item views are only set-like if the values are hashable.
In what cases would be better to use the standard methods keys(), values() and items() ? You mentioned a very important one: backwards compatibility. Also, when you need to have a simple list of all keys, values or items (not a set-like, not an iterator), when you need to modify the returned list without modifying the original dictionary and when you need a snapshot of a dictionary’s keys, values or items at a moment in time, independent of any posterior modifications over the dictionary.
And what about iterkeys(), itervalues() and iteritems()? They’re a suitable alternative when you need a one-shot, constant-space, lazy iterator snapshot of a dictionary’s contents that will tell you if the dictionary got modified while iterating (via a RuntimeError), also they’re very important for backwards compatibility.
Functionality-wise, as you have observed, views are better. Compatibility-wise, they’re worse.
Some performance metrics, taken from Python 2.7.2 on a 64-bit Ubuntu machine:
>>> from timeit import timeit
Dealing with an empty dictionary:
>>> emptydict = {}
>>> timeit(lambda: emptydict.viewkeys())
0.24384498596191406
>>> timeit(lambda: list(emptydict.viewkeys()))
0.4636681079864502
>>> timeit(lambda: emptydict.iterkeys())
0.23939013481140137
>>> timeit(lambda: list(emptydict.iterkeys()))
1.0098130702972412
Constructing the view is slightly more expensive, but consuming the view is significantly faster than the iterator (a bit over twice as fast).
Dealing with a thousand-element dictionary:
>>> fulldict = {i: i for i in xrange(1000)}
>>> timeit(lambda: fulldict.viewkeys())
0.24295306205749512
>>> timeit(lambda: list(fulldict.viewkeys()))
13.447425842285156
>>> timeit(lambda: fulldict.iterkeys())
0.23759889602661133
>>> timeit(lambda: list(fulldict.iterkeys()))
15.45390510559082
Same results, though less marked; constructing the view is very slightly more expensive, but consuming it is quite definitely faster (15% faster).
For fair comparison with list(dict.viewkeys()) and list(dict.iterkeys()), dict.keys() is distinctlyfaster:
>>> timeit(lambda: emptydict.keys())
0.2385849952697754
>>> timeit(lambda: fulldict.keys())
7.842105150222778
Summary: it’s a trade-off; better functionality (which you will rarely use) and performance (which will only very rarely be significant enough to worry you—if you’re caring about such performance matters, you’re probably already in the region of needing to work with numpy/scipy) versus better compatibility and muscle memory usage.
Personally, unless already depending on 2.7-only functionality or unless I am absolutely controlling the runtime environment, I would avoid dictionary views in Python 2 code. Even in these cases, my fingers still want to type iter instead of view, so I let ’em!
In Python 2.7, dictionaries have both an iterkeys method and a viewkeys method (and similar pairs for values and items), giving two different ways to lazily iterate over the keys of the dictionary. The viewkeys method provides the principal feature of iterkeys, with iter(d.viewkeys()) effectively equivalent to d.iterkeys(). Additionally, objects returned viewkeys have convenient set-like features. There thus are strong reasons to favor viewkeys over iterkeys.
What about the other direction? Apart from compatibility with earlier versions of Python, are there any ways in which iterkeys would be preferable to viewkeys? Would anything be lost by just always using viewkeys?
A dictionary view updates as the dictionary does, while an iterator does not necessarily do this.
This means if you work with the view, change the dictionary, then work with the view again, the view will have changed to reflect the dictionary’s new state.
They provide a dynamic view on the dictionary’s entries, which means that when the dictionary changes, the view reflects these changes.
Source
Example:
>>> test = {1: 2, 3: 4}
>>> a = test.iterkeys()
>>> b = test.viewkeys()
>>> del test[1]
>>> test[5] = 6
>>> list(a)
[3, 5]
>>> b
dict_keys([3, 5])
When changes are made to the size, an exception will be thrown:
>>> test = {1: 2, 3: 4}
>>> a = test.iterkeys()
>>> b = test.viewkeys()
>>> test[5] = 6
>>> list(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
RuntimeError: dictionary changed size during iteration
>>> b
dict_keys([1, 3, 5])
It’s also worth noting you can only iterate over a keyiterator once:
>>> test = {1: 2, 3: 4}
>>> a = test.iterkeys()
>>> list(a)
[1, 3]
>>> list(a)
[]
>>> b = test.viewkeys()
>>> b
dict_keys([1, 3])
>>> b
dict_keys([1, 3])
No, there’s no advantage to iterkeys over viewkeys, in the same way that there’s no advantage to keys over either of them. iterkeys is only around for back compatibility. Indeed, in Python 3, viewkeys is the only behaviour that still exists, and it has been renamed to keys – the viewkeys method is actually a backport of the Python 3 behaviour.
As the name (and documentation) indicate, the viewkeys(), viewvalues() and viewitems() methods return a view of the current elements in the dictionary meaning that if the dictionary changes, so does the view; views are lazy. In the general case keys views are set-like, and item views are only set-like if the values are hashable.
In what cases would be better to use the standard methods keys(), values() and items() ? You mentioned a very important one: backwards compatibility. Also, when you need to have a simple list of all keys, values or items (not a set-like, not an iterator), when you need to modify the returned list without modifying the original dictionary and when you need a snapshot of a dictionary’s keys, values or items at a moment in time, independent of any posterior modifications over the dictionary.
And what about iterkeys(), itervalues() and iteritems()? They’re a suitable alternative when you need a one-shot, constant-space, lazy iterator snapshot of a dictionary’s contents that will tell you if the dictionary got modified while iterating (via a RuntimeError), also they’re very important for backwards compatibility.
Functionality-wise, as you have observed, views are better. Compatibility-wise, they’re worse.
Some performance metrics, taken from Python 2.7.2 on a 64-bit Ubuntu machine:
>>> from timeit import timeit
Dealing with an empty dictionary:
>>> emptydict = {}
>>> timeit(lambda: emptydict.viewkeys())
0.24384498596191406
>>> timeit(lambda: list(emptydict.viewkeys()))
0.4636681079864502
>>> timeit(lambda: emptydict.iterkeys())
0.23939013481140137
>>> timeit(lambda: list(emptydict.iterkeys()))
1.0098130702972412
Constructing the view is slightly more expensive, but consuming the view is significantly faster than the iterator (a bit over twice as fast).
Dealing with a thousand-element dictionary:
>>> fulldict = {i: i for i in xrange(1000)}
>>> timeit(lambda: fulldict.viewkeys())
0.24295306205749512
>>> timeit(lambda: list(fulldict.viewkeys()))
13.447425842285156
>>> timeit(lambda: fulldict.iterkeys())
0.23759889602661133
>>> timeit(lambda: list(fulldict.iterkeys()))
15.45390510559082
Same results, though less marked; constructing the view is very slightly more expensive, but consuming it is quite definitely faster (15% faster).
For fair comparison with list(dict.viewkeys()) and list(dict.iterkeys()), dict.keys() is distinctlyfaster:
>>> timeit(lambda: emptydict.keys())
0.2385849952697754
>>> timeit(lambda: fulldict.keys())
7.842105150222778
Summary: it’s a trade-off; better functionality (which you will rarely use) and performance (which will only very rarely be significant enough to worry you—if you’re caring about such performance matters, you’re probably already in the region of needing to work with numpy/scipy) versus better compatibility and muscle memory usage.
Personally, unless already depending on 2.7-only functionality or unless I am absolutely controlling the runtime environment, I would avoid dictionary views in Python 2 code. Even in these cases, my fingers still want to type iter instead of view, so I let ’em!