Merge dataframes in a dictionary
Question:
Say I have an dictionary of dataframes:
{'df1': name color type
Apple Yellow Fruit,
'df2': name color type
Banana Red Fruit,
'df3': name color type
Chocolate Brown Sweet
......}
And I want to merge them all into one like this:
name color type
Apple Red Fruit
Banana Yellow Fruit
Chocolate Brown Sweet
I can do it manually as follows:
merge1=pd.merge('df1','df2')
merge2=pd.merge('merge1','df3')
...
But is there a way to automatically zip through the dictionary and merge?
Any help is appreciated.
Answers:
You can just pass the dict direct and access the values
attribute to concat
:
In [233]:
d
Out[233]:
{'df1': name color type
0 Apple Yellow Fruit, 'df2': name color type
0 Banana Red Fruit, 'df3': name color type
0 Chocolate Brown Sweet}
In [234]:
pd.concat(d.values(), ignore_index=True)
Out[234]:
name color type
0 Banana Red Fruit
1 Apple Yellow Fruit
2 Chocolate Brown Sweet
This assumes that you are just looking to concatenate all the dfs, if you are going to merge then you need to explain what the merge criteria is
You can concatenate all dataframes in a dictionary by doing this
print(d.keys())
df_names = []
for i in d.keys():
temp_df = d[i]
df_names.append(temp_df)
d['merged'] = pd.concat(df_names)
Say I have an dictionary of dataframes:
{'df1': name color type
Apple Yellow Fruit,
'df2': name color type
Banana Red Fruit,
'df3': name color type
Chocolate Brown Sweet
......}
And I want to merge them all into one like this:
name color type
Apple Red Fruit
Banana Yellow Fruit
Chocolate Brown Sweet
I can do it manually as follows:
merge1=pd.merge('df1','df2')
merge2=pd.merge('merge1','df3')
...
But is there a way to automatically zip through the dictionary and merge?
Any help is appreciated.
You can just pass the dict direct and access the values
attribute to concat
:
In [233]:
d
Out[233]:
{'df1': name color type
0 Apple Yellow Fruit, 'df2': name color type
0 Banana Red Fruit, 'df3': name color type
0 Chocolate Brown Sweet}
In [234]:
pd.concat(d.values(), ignore_index=True)
Out[234]:
name color type
0 Banana Red Fruit
1 Apple Yellow Fruit
2 Chocolate Brown Sweet
This assumes that you are just looking to concatenate all the dfs, if you are going to merge then you need to explain what the merge criteria is
You can concatenate all dataframes in a dictionary by doing this
print(d.keys())
df_names = []
for i in d.keys():
temp_df = d[i]
df_names.append(temp_df)
d['merged'] = pd.concat(df_names)