Using an OrderedDict in **kwargs
Question:
Is it possible to pass an OrderedDict instance to a function which uses the **kwargs syntax and retain the ordering?
What I’d like to do is :
def I_crave_order(**kwargs):
for k, v in kwargs.items():
print k, v
example = OrderedDict([('first', 1), ('second', 2), ('third', -1)])
I_crave_order(**example)
>> first 1
>> second 2
>> third -1
However the actual result is:
>> second 2
>> third -1
>> first 1
ie, typical random dict ordering.
I have other uses where setting the order explicitly is good, so I want to keep **kwargs and not just pass the OrderedDict as a regular argument
Answers:
As of Python 3.6, the keyword argument order is preserved. Before 3.6, it is not possible since the OrderedDict gets turned into a dict.
The first thing to realize is that the value you pass in **example does not automatically become the value in **kwargs. Consider this case, where kwargs will only have part of example:
def f(a, **kwargs):
pass
example = {'a': 1, 'b': 2}
f(**example)
Or this case, where it will have more values than those in example:
example = {'b': 2}
f(a=1, c=3, **example)
Or even no overlap at all:
example = {'a': 1}
f(b=2, **example)
So, what you’re asking for doesn’t really make sense.
Still, it might be nice if there were some way to specify that you want an ordered **kwargs, no matter where the keywords came from—explicit keyword args in the order they appear, followed by all of the items of **example in the order they appear in example (which could be arbitrary if example were a dict, but could also be meaningful if it were an OrderedDict).
Defining all the fiddly details, and keeping the performance acceptable, turns out to be harder than it sounds. See PEP 468, and the linked threads, for some discussion on the idea. It seems to have stalled this time around, but if someone picks it up and champions it (and writes a reference implementation for people to play with—which depends on an OrderedDict accessible from the C API, but that will hopefully be there in 3.5+), I suspect it would eventually get into the language.
By the way, note that if this were possible, it would almost certainly be used in the constructor for OrderedDict itself. But if you try that, all you’re doing is freezing some arbitrary order as the permanent order:
>>> d = OrderedDict(a=1, b=2, c=3)
OrderedDict([('a', 1), ('c', 3), ('b', 2)])
Meanwhile, what options do you have?
Well, the obvious option is just to pass example as a normal argument instead of unpacking it:
def f(example):
pass
example = OrderedDict([('a', 1), ('b', 2)])
f(example)
Or, of course, you can use *args and pass the items as tuples, but that’s generally uglier.
There might be some other workarounds in the threads linked from the PEP, but really, none of them are going to be better than this. (Except… IIRC, Li Haoyi came up with a solution based on his MacroPy to pass order-retaining keyword arguments, but I don’t remember the details. MacroPy solutions in general are awesome if you’re willing to use MacroPy and write code that doesn’t quite read like Python, but that isn’t always appropriate…)
When Python encounters the **kwargs construct in a signature, it expects kwargs to be a “mapping”, which means two things: (1) to be able to call kwargs.keys() to obtain an iterable of the keys contained by the mapping, and (2) that kwargs.__getitem__(key) can be called for each key in the iterable returned by keys() and that the resulting value is the desired one to be associated for that key.
Internally, Python will then “transform” whatever the mapping is into a dictionary, sort of like this:
**kwargs -> {key:kwargs[key] for key in kwargs.keys()}
This looks a little silly if you think that kwargs is already a dict — and it would be, since there is no reason to construct a totally equivalent dict from the one that is passed in.
But when kwargs is not necessarily a dict, then it matters to bring its contents down into a suitable default data structure so that the code which carries out the argument unpacking always knows what it’s working with.
So, you can mess with the way a certain data type gets unpackaged, but because of the conversion to dict for the sake of a consistent arg-unpacking-protocol, it just happens that placing guarantees on the order of argument unpacking is not possible (since dict doesn’t keep track of the order in which elements are added). If the language of Python brought **kwargs down into an OrderedDict instead of a dict (meaning that the keys’ order as keyword args would be the order in which they are traversed), then by passing either an OrderedDict or some other data structure where keys() respects some kind of ordering, you could expect a certain ordering on the arguments. It’s just a quirk of the implementation that dict is chosen as the standard, and not some other type of mapping.
Here’s a dumb example of a class that can “be unpacked” but which always treats all unpacked values as 42 (even though they aren’t really):
class MyOrderedDict(object):
def __init__(self, odict):
self._odict = odict
def __repr__(self):
return self._odict.__repr__()
def __getitem__(self, item):
return 42
def __setitem__(self, item, value):
self._odict[item] = value
def keys(self):
return self._odict.keys()
Then define a function to print the unpacked contents:
def foo(**kwargs):
for k, v in kwargs.iteritems():
print k, v
and make a value and try it out:
In [257]: import collections; od = collections.OrderedDict()
In [258]: od['a'] = 1; od['b'] = 2; od['c'] = 3;
In [259]: md = MyOrderedDict(od)
In [260]: print md
OrderedDict([('a', 1), ('b', 2), ('c', 3)])
In [261]: md.keys()
Out[261]: ['a', 'b', 'c']
In [262]: foo(**md)
a 42
c 42
b 42
This customized delivery of key-value pairs (here, dumbly always returning 42) is the extent of your ability to tinker with how **kwargs works in Python.
There is slightly more flexibility for tinkering with how *args gets unpackaged. For more on that see this question: < Does argument unpacking use iteration or item-getting? >.
This is now the default in python 3.6.
Python 3.6.0a4+ (default:d43f819caea7, Sep 8 2016, 13:05:34)
>>> def func(**kw): print(kw.keys())
...
>>> func(a=1, b=2, c=3, d=4, e=5)
dict_keys(['a', 'b', 'c', 'd', 'e']) # expected order
It’s not possible to do it before as noted by the other answers.
Is it possible to pass an OrderedDict instance to a function which uses the **kwargs syntax and retain the ordering?
What I’d like to do is :
def I_crave_order(**kwargs):
for k, v in kwargs.items():
print k, v
example = OrderedDict([('first', 1), ('second', 2), ('third', -1)])
I_crave_order(**example)
>> first 1
>> second 2
>> third -1
However the actual result is:
>> second 2
>> third -1
>> first 1
ie, typical random dict ordering.
I have other uses where setting the order explicitly is good, so I want to keep **kwargs and not just pass the OrderedDict as a regular argument
As of Python 3.6, the keyword argument order is preserved. Before 3.6, it is not possible since the OrderedDict gets turned into a dict.
The first thing to realize is that the value you pass in **example does not automatically become the value in **kwargs. Consider this case, where kwargs will only have part of example:
def f(a, **kwargs):
pass
example = {'a': 1, 'b': 2}
f(**example)
Or this case, where it will have more values than those in example:
example = {'b': 2}
f(a=1, c=3, **example)
Or even no overlap at all:
example = {'a': 1}
f(b=2, **example)
So, what you’re asking for doesn’t really make sense.
Still, it might be nice if there were some way to specify that you want an ordered **kwargs, no matter where the keywords came from—explicit keyword args in the order they appear, followed by all of the items of **example in the order they appear in example (which could be arbitrary if example were a dict, but could also be meaningful if it were an OrderedDict).
Defining all the fiddly details, and keeping the performance acceptable, turns out to be harder than it sounds. See PEP 468, and the linked threads, for some discussion on the idea. It seems to have stalled this time around, but if someone picks it up and champions it (and writes a reference implementation for people to play with—which depends on an OrderedDict accessible from the C API, but that will hopefully be there in 3.5+), I suspect it would eventually get into the language.
By the way, note that if this were possible, it would almost certainly be used in the constructor for OrderedDict itself. But if you try that, all you’re doing is freezing some arbitrary order as the permanent order:
>>> d = OrderedDict(a=1, b=2, c=3)
OrderedDict([('a', 1), ('c', 3), ('b', 2)])
Meanwhile, what options do you have?
Well, the obvious option is just to pass example as a normal argument instead of unpacking it:
def f(example):
pass
example = OrderedDict([('a', 1), ('b', 2)])
f(example)
Or, of course, you can use *args and pass the items as tuples, but that’s generally uglier.
There might be some other workarounds in the threads linked from the PEP, but really, none of them are going to be better than this. (Except… IIRC, Li Haoyi came up with a solution based on his MacroPy to pass order-retaining keyword arguments, but I don’t remember the details. MacroPy solutions in general are awesome if you’re willing to use MacroPy and write code that doesn’t quite read like Python, but that isn’t always appropriate…)
When Python encounters the **kwargs construct in a signature, it expects kwargs to be a “mapping”, which means two things: (1) to be able to call kwargs.keys() to obtain an iterable of the keys contained by the mapping, and (2) that kwargs.__getitem__(key) can be called for each key in the iterable returned by keys() and that the resulting value is the desired one to be associated for that key.
Internally, Python will then “transform” whatever the mapping is into a dictionary, sort of like this:
**kwargs -> {key:kwargs[key] for key in kwargs.keys()}
This looks a little silly if you think that kwargs is already a dict — and it would be, since there is no reason to construct a totally equivalent dict from the one that is passed in.
But when kwargs is not necessarily a dict, then it matters to bring its contents down into a suitable default data structure so that the code which carries out the argument unpacking always knows what it’s working with.
So, you can mess with the way a certain data type gets unpackaged, but because of the conversion to dict for the sake of a consistent arg-unpacking-protocol, it just happens that placing guarantees on the order of argument unpacking is not possible (since dict doesn’t keep track of the order in which elements are added). If the language of Python brought **kwargs down into an OrderedDict instead of a dict (meaning that the keys’ order as keyword args would be the order in which they are traversed), then by passing either an OrderedDict or some other data structure where keys() respects some kind of ordering, you could expect a certain ordering on the arguments. It’s just a quirk of the implementation that dict is chosen as the standard, and not some other type of mapping.
Here’s a dumb example of a class that can “be unpacked” but which always treats all unpacked values as 42 (even though they aren’t really):
class MyOrderedDict(object):
def __init__(self, odict):
self._odict = odict
def __repr__(self):
return self._odict.__repr__()
def __getitem__(self, item):
return 42
def __setitem__(self, item, value):
self._odict[item] = value
def keys(self):
return self._odict.keys()
Then define a function to print the unpacked contents:
def foo(**kwargs):
for k, v in kwargs.iteritems():
print k, v
and make a value and try it out:
In [257]: import collections; od = collections.OrderedDict()
In [258]: od['a'] = 1; od['b'] = 2; od['c'] = 3;
In [259]: md = MyOrderedDict(od)
In [260]: print md
OrderedDict([('a', 1), ('b', 2), ('c', 3)])
In [261]: md.keys()
Out[261]: ['a', 'b', 'c']
In [262]: foo(**md)
a 42
c 42
b 42
This customized delivery of key-value pairs (here, dumbly always returning 42) is the extent of your ability to tinker with how **kwargs works in Python.
There is slightly more flexibility for tinkering with how *args gets unpackaged. For more on that see this question: < Does argument unpacking use iteration or item-getting? >.
This is now the default in python 3.6.
Python 3.6.0a4+ (default:d43f819caea7, Sep 8 2016, 13:05:34)
>>> def func(**kw): print(kw.keys())
...
>>> func(a=1, b=2, c=3, d=4, e=5)
dict_keys(['a', 'b', 'c', 'd', 'e']) # expected order
It’s not possible to do it before as noted by the other answers.