Repeat rows in a pandas DataFrame based on column value
Question:
I have the following df:
code . role . persons
123 . Janitor . 3
123 . Analyst . 2
321 . Vallet . 2
321 . Auditor . 5
The first line means that I have 3 persons with the role Janitors.
My problem is that I would need to have one line for each person. My df should look like this:
df:
code . role . persons
123 . Janitor . 3
123 . Janitor . 3
123 . Janitor . 3
123 . Analyst . 2
123 . Analyst . 2
321 . Vallet . 2
321 . Vallet . 2
321 . Auditor . 5
321 . Auditor . 5
321 . Auditor . 5
321 . Auditor . 5
321 . Auditor . 5
How could I do that using pandas?
Answers:
reindex
+ repeat
df.reindex(df.index.repeat(df.persons))
Out[951]:
code . role ..1 persons
0 123 . Janitor . 3
0 123 . Janitor . 3
0 123 . Janitor . 3
1 123 . Analyst . 2
1 123 . Analyst . 2
2 321 . Vallet . 2
2 321 . Vallet . 2
3 321 . Auditor . 5
3 321 . Auditor . 5
3 321 . Auditor . 5
3 321 . Auditor . 5
3 321 . Auditor . 5
PS: you can add.reset_index(drop=True)
to get the new index
Wen’s solution is really nice and intuitive, however it will fail for duplicate rows by throwing ValueError: cannot reindex from a duplicate axis
.
Here’s an alternative which avoids this by calling repeat
on df.values
.
df
code role persons
0 123 Janitor 3
1 123 Analyst 2
2 321 Vallet 2
3 321 Auditor 5
pd.DataFrame(df.values.repeat(df.persons, axis=0), columns=df.columns)
code role persons
0 123 Janitor 3
1 123 Janitor 3
2 123 Janitor 3
3 123 Analyst 2
4 123 Analyst 2
5 321 Vallet 2
6 321 Vallet 2
7 321 Auditor 5
8 321 Auditor 5
9 321 Auditor 5
10 321 Auditor 5
11 321 Auditor 5
Not enough reputation to comment, but building on @cs95’s answer and @lmiguelvargasf’s comment, one can preserve dtypes with:
pd.DataFrame(
df.values.repeat(df.persons, axis=0),
columns=df.columns,
).astype(df.dtypes)
You can apply the Series method repeat
:
df = pd.DataFrame({'col1': [2, 3],
'col2': ['a', 'b'],
'col3': [20, 30]})
df.apply(lambda x: x.repeat(df['col1']))
# df.apply(pd.Series.repeat, repeats=df['col1'])
or the numpy function repeat
:
df.apply(np.repeat, repeats=df['col1'])
Output:
col1 col2 col3
0 2 a 20
0 2 a 20
1 3 b 30
1 3 b 30
1 3 b 30
I have the following df:
code . role . persons
123 . Janitor . 3
123 . Analyst . 2
321 . Vallet . 2
321 . Auditor . 5
The first line means that I have 3 persons with the role Janitors.
My problem is that I would need to have one line for each person. My df should look like this:
df:
code . role . persons
123 . Janitor . 3
123 . Janitor . 3
123 . Janitor . 3
123 . Analyst . 2
123 . Analyst . 2
321 . Vallet . 2
321 . Vallet . 2
321 . Auditor . 5
321 . Auditor . 5
321 . Auditor . 5
321 . Auditor . 5
321 . Auditor . 5
How could I do that using pandas?
reindex
+ repeat
df.reindex(df.index.repeat(df.persons))
Out[951]:
code . role ..1 persons
0 123 . Janitor . 3
0 123 . Janitor . 3
0 123 . Janitor . 3
1 123 . Analyst . 2
1 123 . Analyst . 2
2 321 . Vallet . 2
2 321 . Vallet . 2
3 321 . Auditor . 5
3 321 . Auditor . 5
3 321 . Auditor . 5
3 321 . Auditor . 5
3 321 . Auditor . 5
PS: you can add.reset_index(drop=True)
to get the new index
Wen’s solution is really nice and intuitive, however it will fail for duplicate rows by throwing ValueError: cannot reindex from a duplicate axis
.
Here’s an alternative which avoids this by calling repeat
on df.values
.
df
code role persons
0 123 Janitor 3
1 123 Analyst 2
2 321 Vallet 2
3 321 Auditor 5
pd.DataFrame(df.values.repeat(df.persons, axis=0), columns=df.columns)
code role persons
0 123 Janitor 3
1 123 Janitor 3
2 123 Janitor 3
3 123 Analyst 2
4 123 Analyst 2
5 321 Vallet 2
6 321 Vallet 2
7 321 Auditor 5
8 321 Auditor 5
9 321 Auditor 5
10 321 Auditor 5
11 321 Auditor 5
Not enough reputation to comment, but building on @cs95’s answer and @lmiguelvargasf’s comment, one can preserve dtypes with:
pd.DataFrame(
df.values.repeat(df.persons, axis=0),
columns=df.columns,
).astype(df.dtypes)
You can apply the Series method repeat
:
df = pd.DataFrame({'col1': [2, 3],
'col2': ['a', 'b'],
'col3': [20, 30]})
df.apply(lambda x: x.repeat(df['col1']))
# df.apply(pd.Series.repeat, repeats=df['col1'])
or the numpy function repeat
:
df.apply(np.repeat, repeats=df['col1'])
Output:
col1 col2 col3
0 2 a 20
0 2 a 20
1 3 b 30
1 3 b 30
1 3 b 30