how to concat two data frames with different column names in pandas? – python

Question

df1 = pd.DataFrame({'a':[1,2,3],'x':[4,5,6],'y':[7,8,9]})
df2 = pd.DataFrame({'b':[10,11,12],'x':[13,14,15],'y':[16,17,18]})

I’m trying to merge the two data frames using the keys from the df1. I think I should use pd.merge for this, but I how can I tell pandas to place the values in the b column of df2 in the a column of df1. This is the output I’m trying to achieve:

    a   x   y
0   1   4   7
1   2   5   8
2   3   6   9
3   10  13  16
4   11  14  17
5   12  15  18

Asked By: HappyPy

||

Source

Answer 1

Just use concat and rename the column for df2 so it aligns:

In [92]:
pd.concat([df1,df2.rename(columns={'b':'a'})], ignore_index=True)

Out[92]:
    a   x   y
0   1   4   7
1   2   5   8
2   3   6   9
3  10  13  16
4  11  14  17
5  12  15  18

similarly you can use merge but you’d need to rename the column as above:

In [103]:
df1.merge(df2.rename(columns={'b':'a'}),how='outer')

Out[103]:
    a   x   y
0   1   4   7
1   2   5   8
2   3   6   9
3  10  13  16
4  11  14  17
5  12  15  18

Answered By: EdChum

Answer 2

Use numpy to concatenate the dataframes, so you don’t have to rename all of the columns (or explicitly ignore indexes). np.concatenate also works on an arbitrary number of dataframes.

df = pd.DataFrame( np.concatenate( (df1.values, df2.values), axis=0 ) )
df.columns = [ 'a', 'x', 'y' ]
df

Answered By: scottlittle

Answer 3

You can rename columns and then use functions append or concat:

df2.columns = df1.columns
df1.append(df2, ignore_index=True)
# pd.concat([df1, df2], ignore_index=True)

You can also concatenate both dataframes with vstack from numpy and convert the resulting ndarray to dataframe:

pd.DataFrame(np.vstack([df1, df2]), columns=df1.columns)

Answered By: Mykola Zotko

how to concat two data frames with different column names in pandas? – python

Question:

Answers: