Mapping over values in a python dictionary
Question:
Given a dictionary { k1: v1, k2: v2 ... } I want to get { k1: f(v1), k2: f(v2) ... } provided I pass a function f.
Is there any such built in function? Or do I have to do
dict([(k, f(v)) for (k, v) in my_dictionary.iteritems()])
Ideally I would just write
my_dictionary.map_values(f)
or
my_dictionary.mutate_values_with(f)
That is, it doesn’t matter to me if the original dictionary is mutated or a copy is created.
Answers:
There is no such function; the easiest way to do this is to use a dict comprehension:
my_dictionary = {k: f(v) for k, v in my_dictionary.items()}
In python 2.7, use the .iteritems() method instead of .items() to save memory. The dict comprehension syntax wasn’t introduced until python 2.7.
Note that there is no such method on lists either; you’d have to use a list comprehension or the map() function.
As such, you could use the map() function for processing your dict as well:
my_dictionary = dict(map(lambda kv: (kv[0], f(kv[1])), my_dictionary.iteritems()))
but that’s not that readable, really.
These toolz are great for this kind of simple yet repetitive logic.
http://toolz.readthedocs.org/en/latest/api.html#toolz.dicttoolz.valmap
Gets you right where you want to be.
import toolz
def f(x):
return x+1
toolz.valmap(f, my_list)
You can do this in-place, rather than create a new dict, which may be preferable for large dictionaries (if you do not need a copy).
def mutate_dict(f,d):
for k, v in d.iteritems():
d[k] = f(v)
my_dictionary = {'a':1, 'b':2}
mutate_dict(lambda x: x+1, my_dictionary)
results in my_dictionary containing:
{'a': 2, 'b': 3}
While my original answer missed the point (by trying to solve this problem with the solution to Accessing key in factory of defaultdict), I have reworked it to propose an actual solution to the present question.
Here it is:
class walkableDict(dict):
def walk(self, callback):
try:
for key in self:
self[key] = callback(self[key])
except TypeError:
return False
return True
Usage:
>>> d = walkableDict({ k1: v1, k2: v2 ... })
>>> d.walk(f)
The idea is to subclass the original dict to give it the desired functionality: “mapping” a function over all the values.
The plus point is that this dictionary can be used to store the original data as if it was a dict, while transforming any data on request with a callback.
Of course, feel free to name the class and the function the way you want (the name chosen in this answer is inspired by PHP’s array_walk() function).
Note: Neither the try–except block nor the return statements are mandatory for the functionality, they are there to further mimic the behavior of the PHP’s array_walk.
Due to PEP-0469 which renamed iteritems() to items() and PEP-3113 which removed Tuple parameter unpacking, in Python 3.x you should write Martijn Pieters♦ answer like this:
my_dictionary = dict(map(lambda item: (item[0], f(item[1])), my_dictionary.items()))
Just came accross this use case. I implemented gens’s answer, adding a recursive approach for handling values that are also dicts:
def mutate_dict_in_place(f, d):
for k, v in d.iteritems():
if isinstance(v, dict):
mutate_dict_in_place(f, v)
else:
d[k] = f(v)
# Exemple handy usage
def utf8_everywhere(d):
mutate_dict_in_place((
lambda value:
value.decode('utf-8')
if isinstance(value, bytes)
else value
),
d
)
my_dict = {'a': b'byte1', 'b': {'c': b'byte2', 'd': b'byte3'}}
utf8_everywhere(my_dict)
print(my_dict)
This can be useful when dealing with json or yaml files that encode strings as bytes in Python 2
To avoid doing indexing from inside lambda, like:
rval = dict(map(lambda kv : (kv[0], ' '.join(kv[1])), rval.iteritems()))
You can also do:
rval = dict(map(lambda(k,v) : (k, ' '.join(v)), rval.iteritems()))
- My way to map over dictionary
def f(x): return x+2
bill = {"Alice": 20, "Bob": 10}
d = {map(lambda x: f(x),bill.values())}
print('d: ',dict(d))
Results
: d: {22: 12}
- Map over iterable in values within dictionary
bills = {"Alice": [20, 15, 30], "Bob": [10, 35]}
d= {map(lambda v: sum(v),bills.values())}
g= dict(map(lambda v: (v[0],sum(v[1])),bills.items()))
# prints
print('d: ',dict(d))
print('g: ',g)
Results
d: {65: 45}
g: {'Alice': 65, 'Bob': 45}
Given a dictionary { k1: v1, k2: v2 ... } I want to get { k1: f(v1), k2: f(v2) ... } provided I pass a function f.
Is there any such built in function? Or do I have to do
dict([(k, f(v)) for (k, v) in my_dictionary.iteritems()])
Ideally I would just write
my_dictionary.map_values(f)
or
my_dictionary.mutate_values_with(f)
That is, it doesn’t matter to me if the original dictionary is mutated or a copy is created.
There is no such function; the easiest way to do this is to use a dict comprehension:
my_dictionary = {k: f(v) for k, v in my_dictionary.items()}
In python 2.7, use the .iteritems() method instead of .items() to save memory. The dict comprehension syntax wasn’t introduced until python 2.7.
Note that there is no such method on lists either; you’d have to use a list comprehension or the map() function.
As such, you could use the map() function for processing your dict as well:
my_dictionary = dict(map(lambda kv: (kv[0], f(kv[1])), my_dictionary.iteritems()))
but that’s not that readable, really.
These toolz are great for this kind of simple yet repetitive logic.
http://toolz.readthedocs.org/en/latest/api.html#toolz.dicttoolz.valmap
Gets you right where you want to be.
import toolz
def f(x):
return x+1
toolz.valmap(f, my_list)
You can do this in-place, rather than create a new dict, which may be preferable for large dictionaries (if you do not need a copy).
def mutate_dict(f,d):
for k, v in d.iteritems():
d[k] = f(v)
my_dictionary = {'a':1, 'b':2}
mutate_dict(lambda x: x+1, my_dictionary)
results in my_dictionary containing:
{'a': 2, 'b': 3}
While my original answer missed the point (by trying to solve this problem with the solution to Accessing key in factory of defaultdict), I have reworked it to propose an actual solution to the present question.
Here it is:
class walkableDict(dict):
def walk(self, callback):
try:
for key in self:
self[key] = callback(self[key])
except TypeError:
return False
return True
Usage:
>>> d = walkableDict({ k1: v1, k2: v2 ... })
>>> d.walk(f)
The idea is to subclass the original dict to give it the desired functionality: “mapping” a function over all the values.
The plus point is that this dictionary can be used to store the original data as if it was a dict, while transforming any data on request with a callback.
Of course, feel free to name the class and the function the way you want (the name chosen in this answer is inspired by PHP’s array_walk() function).
Note: Neither the try–except block nor the return statements are mandatory for the functionality, they are there to further mimic the behavior of the PHP’s array_walk.
Due to PEP-0469 which renamed iteritems() to items() and PEP-3113 which removed Tuple parameter unpacking, in Python 3.x you should write Martijn Pieters♦ answer like this:
my_dictionary = dict(map(lambda item: (item[0], f(item[1])), my_dictionary.items()))
Just came accross this use case. I implemented gens’s answer, adding a recursive approach for handling values that are also dicts:
def mutate_dict_in_place(f, d):
for k, v in d.iteritems():
if isinstance(v, dict):
mutate_dict_in_place(f, v)
else:
d[k] = f(v)
# Exemple handy usage
def utf8_everywhere(d):
mutate_dict_in_place((
lambda value:
value.decode('utf-8')
if isinstance(value, bytes)
else value
),
d
)
my_dict = {'a': b'byte1', 'b': {'c': b'byte2', 'd': b'byte3'}}
utf8_everywhere(my_dict)
print(my_dict)
This can be useful when dealing with json or yaml files that encode strings as bytes in Python 2
To avoid doing indexing from inside lambda, like:
rval = dict(map(lambda kv : (kv[0], ' '.join(kv[1])), rval.iteritems()))
You can also do:
rval = dict(map(lambda(k,v) : (k, ' '.join(v)), rval.iteritems()))
- My way to map over dictionary
def f(x): return x+2
bill = {"Alice": 20, "Bob": 10}
d = {map(lambda x: f(x),bill.values())}
print('d: ',dict(d))
Results
: d: {22: 12}
- Map over iterable in values within dictionary
bills = {"Alice": [20, 15, 30], "Bob": [10, 35]}
d= {map(lambda v: sum(v),bills.values())}
g= dict(map(lambda v: (v[0],sum(v[1])),bills.items()))
# prints
print('d: ',dict(d))
print('g: ',g)
Results
d: {65: 45}
g: {'Alice': 65, 'Bob': 45}