How to concatenate rows side by side in pandas

Question:

I want to combine the five rows of the same dataset into a single dataset
I have 700 rows and i want to combining every five rows

      A  B  C  D  E  F   G
1     10,11,12,13,14,15,16    
2     17,18,19,20,21,22,23    
3     24,25,26,27,28,29,30      
4     31,32,33,34,35,36,37    
5     38,39,40,41,42,43,44
.
.
.
.
.
700

After combining the first five rows.. My first row should look like this:

        A  B  C  D  E  F  G  A  B  C  D  E  F  G  A  B  C  D  E  F  G  A  B  C  D  E  F  G  A  B  C  D  E  F  G
                                                                         
    1  10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44
Asked By: bigci10

||

Answers:

the easiest way is to convert your dataframe to a numpy array, reshape it then cast it back to a new dataframe.

Edit:

data= # your dataframe
new_dataframe=pd.DataFrame(data.to_numpy().reshape(len(data)//5,-1),columns=np.tile(data.columns,5))
Answered By: Kaoutar

You can do:

cols = [col for v in [df.columns.tolist()]*len(df) for col in v]
dfs = [df[i:min(i+5,len(df))].reset_index(drop=True) for i in range(0,len(df),5)]
df2 = pd.concat([pd.DataFrame(df.stack()).T for df in dfs])
df2.columns = cols
df2.reset_index(drop=True, inplace=True)
Answered By: SomeDude

Try this using arange() with floordiv to get groups by every 5, then creating a new df with the groups. This should work even if your df is not divisible by 5.

l = 5
(df.groupby(np.arange(len(df.index))//l)
 .apply(lambda x: pd.DataFrame([x.to_numpy().ravel()]))
 .set_axis(df.columns.tolist() * l,axis=1)
 .reset_index(drop=True))

or

(df.groupby(np.arange(len(df.index))//5)
.apply(lambda x: x.reset_index(drop=True).stack())
.droplevel(0,axis=1))

Output:

   A  B  C  D  E  F  G  A  B  C  ...  E  F  G  A  B  C  D  E  F  G
0  9  0  3  2  6  2  9  1  7  5  ...  2  5  9  5  4  9  7  3  8  9
1  9  5  0  8  1  5  8  7  7  7  ...  6  3  5  5  2  3  9  7  5  6
Answered By: rhug123

see if this helps answer your question
unstack turns the columns into rows, and once we have the data in a column, we just need it transposed. reset_index makes the resulting series into a dataframe. the original columns names are made into an index, so when we transpose we have the columns as you had stated in your columns.

df.unstack().reset_index().set_index('level_0')[[0]].T
level_0 A   A   A   A   A   B   B   B   B   B   ... F   F   F   F   F   G   G   G   G   G
0   10  17  24  31  38  11  18  25  32  39  ... 15  22  29  36  43  16  23  30  37  44

vote and/or accept if the answer helps

Answered By: Naveed

If you can guarantee that the total number of rows you have is a multiple of 5, dipping into numpy will be the most efficient way to solve this problem:

import numpy as np
import pandas as pd

data = np.arange(70).reshape(-1, 7)
df = pd.DataFrame(data, columns=[*'ABCDEFG'])

print(df)
    A   B   C   D   E   F   G
0   0   1   2   3   4   5   6
1   7   8   9  10  11  12  13
2  14  15  16  17  18  19  20
3  21  22  23  24  25  26  27
4  28  29  30  31  32  33  34
5  35  36  37  38  39  40  41
6  42  43  44  45  46  47  48
7  49  50  51  52  53  54  55
8  56  57  58  59  60  61  62
9  63  64  65  66  67  68  69

out = pd.DataFrame(
    df.to_numpy().reshape(-1, df.shape[1] * 5),
    columns=[*df.columns] * 5
)

print(out)
    A   B   C   D   E   F   G   A   B   C   D   E   F  ...   B   C   D   E   F   G   A   B   C   D   E   F   G
0   0   1   2   3   4   5   6   7   8   9  10  11  12  ...  22  23  24  25  26  27  28  29  30  31  32  33  34
1  35  36  37  38  39  40  41  42  43  44  45  46  47  ...  57  58  59  60  61  62  63  64  65  66  67  68  69

[2 rows x 35 columns]

Answered By: Cameron Riddell

Stacking and unstacking data in pandas

Data in tables are often presented multiple ways. Long form ("tidy data") refers to data that are stacked in a couple of columns. One of the columns will have categorical indicators about the values. In contrast, wide form ("stacked data") is where each category has it’s own column.

In your example, you present the wide form of data, and you’re trying to get it into long form. The pandas.melt, pandas.groupby, pandas.pivot, pandas.stack, pandas.unstack, and pandas.reset_index are the functions that help convert between these forms.

Start with your original dataframe:

df = pd.DataFrame({
   'A' : [10, 17, 24, 31, 38],
   'B' : [11, 18, 25, 32, 39],
   'C' : [12, 19, 26, 33, 40],
   'D' : [13, 20, 27, 34, 41],
   'E' : [14, 21, 28, 35, 42],
   'F' : [15, 22, 29, 36, 43],
   'G' : [16, 23, 30, 37, 44]})

    A   B   C   D   E   F   G
0   10  11  12  13  14  15  16
1   17  18  19  20  21  22  23
2   24  25  26  27  28  29  30
3   31  32  33  34  35  36  37
4   38  39  40  41  42  43  44

Use pandas.melt to convert it to long form, then sort to get it how you requested the data: The ignore index option helps us to get it back to wide form later.

melted_df = df.melt(ignore_index=False).sort_values(by='value')


variable    value
0   A   10
0   B   11
0   C   12
0   D   13
0   E   14
0   F   15
0   G   16
1   A   17
1   B   18
...

Use groupby, unstack, and reset_index to convert it back to wide form. This is often a much more difficult process that relies on grouping by the value stacked column, other columns, index, and stacked variable and then unstacking and resetting the index.

(melted_df
    .reset_index() # puts the index values into a column called 'index'
    .groupby(['index','variable']) #groups by the index and the variable
    .value  #selects the value column in each of the groupby objects
    .mean() #since there is only one item per group, it only aggregates one item
    .unstack() #this sets the first item of the multi-index to columns
    .reset_index() #fix the index
    .set_index('index') #set index
)
    A   B   C   D   E   F   G                           
0   10  11  12  13  14  15  16
1   17  18  19  20  21  22  23
2   24  25  26  27  28  29  30
3   31  32  33  34  35  36  37
4   38  39  40  41  42  43  44

This stuff can be quite difficult and requires trial and error. I would recommend making a smaller version of your problems and mess with them. This way you can figure out how the functions are working.

Answered By: Coup
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.