pd.concat not stacking columns with same names

Question:

I’m in awe that this isn’t working because I’ve done this a hundred times. I want to stack two dataframes vertically and pandas is creating duplicate columns and refusing to put the data in the right columns.

df1 looks like this:

enter image description here

df2 looks like this:

enter image description here

then I run this:

frames = [df1,df2]
final = pd.concat(frames, ignore_index = True, axis = 0)
final

and get 6 columns instead of 3 like this:

enter image description here

I have no idea why two dataframes with identical column names and data types would not simply stack on top of each other. Any help appreciated.

Thanks.

update:

Big Thanks to @Naveed there was trailing whitespace in one of the dataframe’s columns.

Here is the final fix:

df2.columns = [x.strip() for x in df2.columns]
frames = [df1,df2]
final = pd.concat(frames,ignore_index = True, axis = 0)
final
Asked By: Justin Benfit

||

Answers:

Try

check the column names, there might be white spaces that results in mis-alignment of column after the concat.

display(df1.columns, df2.columns)
#  make axis=0 and remove ignore_index

final = pd.concat(frames,   axis = 0)
final
Answered By: Naveed
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.