How can I concatenate vertically multiple columns in pandas
Question:
I’m trying to add a new column to a dataframe, and fill up that column with multiple other columns in the dataframe concatenated together. How can I do this with the fact that this new column will have a different length than the rest of the columns in the dataframe?
For example:
df = pd.DataFrame([[1, 2], [1, 3], [4, 6]], columns=['A', 'B'])
I would want to create a column C in the dataframe that reads 1,1,4,2,3,6 (except vertically)
print (df)
A B C
0 1 2 1
1 1 3 1
2 4 6 4
3 2
4 3
5 6
Answers:
How about this:
df1 = pd.DataFrame.from_dict({'A':[1,1,4],'B':[2,3,6]})
concatvalues = np.concatenate([df1.A.values,df1.B.values])
df2 = pd.concat([df1,pd.DataFrame(concatvalues)], ignore_index=True, axis=1)
df2.columns = np.append(df1.columns.values, "concat")
print(df2)
prints
A B concat
0 1.0 2.0 1
1 1.0 3.0 1
2 4.0 6.0 4
3 NaN NaN 2
4 NaN NaN 3
5 NaN NaN 6
Slightly modified Anton’s solution (will work for any number of columns):
In [99]: df = pd.DataFrame(np.random.randint(0,10,(3,4)), columns=list('ABCD'))
In [100]: df
Out[100]:
A B C D
0 9 6 9 6
1 1 2 0 8
2 5 0 4 8
In [105]: pd.concat([df, df.T.stack().reset_index(name='new')['new']], axis=1)
Out[105]:
A B C D new
0 9.0 6.0 9.0 6.0 9
1 1.0 2.0 0.0 8.0 1
2 5.0 0.0 4.0 8.0 5
3 NaN NaN NaN NaN 6
4 NaN NaN NaN NaN 2
5 NaN NaN NaN NaN 0
6 NaN NaN NaN NaN 9
7 NaN NaN NaN NaN 0
8 NaN NaN NaN NaN 4
9 NaN NaN NaN NaN 6
10 NaN NaN NaN NaN 8
11 NaN NaN NaN NaN 8
If you have a long list of columns that you need to stack vertically – you can use the following syntax, rather than naming them all inside pd.concat():
pd.concat([df.loc[:, col] for col in df.columns], axis = 0, ignore_index=True)
I’m trying to add a new column to a dataframe, and fill up that column with multiple other columns in the dataframe concatenated together. How can I do this with the fact that this new column will have a different length than the rest of the columns in the dataframe?
For example:
df = pd.DataFrame([[1, 2], [1, 3], [4, 6]], columns=['A', 'B'])
I would want to create a column C in the dataframe that reads 1,1,4,2,3,6 (except vertically)
print (df)
A B C
0 1 2 1
1 1 3 1
2 4 6 4
3 2
4 3
5 6
How about this:
df1 = pd.DataFrame.from_dict({'A':[1,1,4],'B':[2,3,6]})
concatvalues = np.concatenate([df1.A.values,df1.B.values])
df2 = pd.concat([df1,pd.DataFrame(concatvalues)], ignore_index=True, axis=1)
df2.columns = np.append(df1.columns.values, "concat")
print(df2)
prints
A B concat
0 1.0 2.0 1
1 1.0 3.0 1
2 4.0 6.0 4
3 NaN NaN 2
4 NaN NaN 3
5 NaN NaN 6
Slightly modified Anton’s solution (will work for any number of columns):
In [99]: df = pd.DataFrame(np.random.randint(0,10,(3,4)), columns=list('ABCD'))
In [100]: df
Out[100]:
A B C D
0 9 6 9 6
1 1 2 0 8
2 5 0 4 8
In [105]: pd.concat([df, df.T.stack().reset_index(name='new')['new']], axis=1)
Out[105]:
A B C D new
0 9.0 6.0 9.0 6.0 9
1 1.0 2.0 0.0 8.0 1
2 5.0 0.0 4.0 8.0 5
3 NaN NaN NaN NaN 6
4 NaN NaN NaN NaN 2
5 NaN NaN NaN NaN 0
6 NaN NaN NaN NaN 9
7 NaN NaN NaN NaN 0
8 NaN NaN NaN NaN 4
9 NaN NaN NaN NaN 6
10 NaN NaN NaN NaN 8
11 NaN NaN NaN NaN 8
If you have a long list of columns that you need to stack vertically – you can use the following syntax, rather than naming them all inside pd.concat():
pd.concat([df.loc[:, col] for col in df.columns], axis = 0, ignore_index=True)