Pandas: Convert DataFrame with MultiIndex to dict

Question

Another novice pandas question. I want to convert a DataFrame to a dictionary, but in a way different from what is offered by the DataFrame.to_dict() function. Explanation by example:

df = pd.DataFrame({'co':['DE','DE','FR','FR'],
                   'tp':['Lake','Forest','Lake','Forest'],
                   'area':[10,20,30,40],
                   'count':[7,5,2,3]})
df = df.set_index(['co','tp'])

Before:

           area  count
co tp
DE Lake      10      7
   Forest    20      5
FR Lake      30      2
   Forest    40      3

After:

{('DE', 'Lake', 'area'): 10,
 ('DE', 'Lake', 'count'): 7,
 ('DE', 'Forest', 'area'): 20,
 ...
 ('FR', 'Forest', 'count'): 3 }

The dict keys should be tuples consisting of the index row + column title, while the dict values should be the individual DataFrame values. For the example above, I managed to find this expression:

after = {(r[0],r[1],c):df.ix[r,c] for c in df.columns for r in df.index}

How can I generalize this code to work for MultiIndices with N levels (instead of 2)?

Answer

Thanks to DSM’s answer, I found that I actually just need to use tuple concatenation r+(c,) and my 2-dimensional loop above becomes N-dimensional:

after = {r + (c,): df.ix[r,c] for c in df.columns for r in df.index}

Asked By: ojdo

||

Source

Answer 1

How about:

>>> df
           area  count
co tp                 
DE Lake      10      7
   Forest    20      5
FR Lake      30      2
   Forest    40      3
>>> after = {r + (k,): v for r, kv in df.iterrows() for k,v in kv.to_dict().items()}
>>> import pprint
>>> pprint.pprint(after)
{('DE', 'Forest', 'area'): 20,
 ('DE', 'Forest', 'count'): 5,
 ('DE', 'Lake', 'area'): 10,
 ('DE', 'Lake', 'count'): 7,
 ('FR', 'Forest', 'area'): 40,
 ('FR', 'Forest', 'count'): 3,
 ('FR', 'Lake', 'area'): 30,
 ('FR', 'Lake', 'count'): 2}

Answered By: DSM

Answer 2

df.stack().to_dict()

out：

{('DE', 'Lake', 'area'): 10,
 ('DE', 'Lake', 'count'): 7,
 ('DE', 'Forest', 'area'): 20,
 ('DE', 'Forest', 'count'): 5,
 ('FR', 'Lake', 'area'): 30,
 ('FR', 'Lake', 'count'): 2,
 ('FR', 'Forest', 'area'): 40,
 ('FR', 'Forest', 'count'): 3}

Answered By: G.G

Pandas: Convert DataFrame with MultiIndex to dict

Question:

Answers: