python pandas select both head and tail

Question:

For a DataFrame in Pandas, how can I select both the first 5 values and last 5 values?

For example

In [11]: df
Out[11]: 
        A  B  C
2012-11-29  0  0  0
2012-11-30  1  1  1
2012-12-01  2  2  2
2012-12-02  3  3  3
2012-12-03  4  4  4
2012-12-04  5  5  5
2012-12-05  6  6  6
2012-12-06  7  7  7
2012-12-07  8  8  8
2012-12-08  9  9  9

How to show the first two and the last two rows?

Asked By: fu xue

||

Answers:

You can use iloc with numpy.r_:

print (np.r_[0:2, -2:0])
[ 0  1 -2 -1]

df = df.iloc[np.r_[0:2, -2:0]]
print (df)
            A  B  C
2012-11-29  0  0  0
2012-11-30  1  1  1
2012-12-07  8  8  8
2012-12-08  9  9  9

df = df.iloc[np.r_[0:4, -4:0]]
print (df)
            A  B  C
2012-11-29  0  0  0
2012-11-30  1  1  1
2012-12-01  2  2  2
2012-12-02  3  3  3
2012-12-05  6  6  6
2012-12-06  7  7  7
2012-12-07  8  8  8
2012-12-08  9  9  9
Answered By: jezrael

You can use df.head(5) and df.tail(5) to get first five and last five.
Optionally you can create new data frame and append() head and tail:

new_df = df.tail(5)
new_df = new_df.append(df.head(5))
Answered By: Linas Fx

Small simple function:

def ends(df, x=5):
    return df.head(x).append(df.tail(x))

And use like so:

df = pd.DataFrame(np.random.rand(15,6))
ends(df,2)

I actually use this so much, I think it would be a great feature to add to pandas. (No features are to be added to pandas.DataFrame core API) I add it after import like so:

import pandas as pd
def ends(df, x=5):
    return df.head(x).append(df.tail(x))
setattr(pd.DataFrame,'ends',ends)

Use like so:

import numpy as np
df = pd.DataFrame(np.random.rand(15,6))
df.ends(2)
Answered By: ic_fl2

Associated with Linas Fx.

Defining below

pd.DataFrame.less = lambda df, n=10: df.head(n//2).append(df.tail(n//2))

then you can type only df.less()

It’s same as type df.head().append(df.tail())

If you type df.less(2), the result is same as df.head(1).append(df.tail(1))

Answered By: You Oneandzero

Not quite the same question but if you just want to show the top / bottom 5 rows (eg with display in jupyter or regular print, there’s potentially a simpler way than this if you use the pd.option_context context.

#make 100 3d random numbers
df = pd.DataFrame(np.random.randn(100,3))

# sort them by their axis sum
df = df.loc[df.sum(axis=1).index]

with pd.option_context('display.max_rows',10):
    print(df)

Outputs:

           0         1         2
0  -0.649105 -0.413335  0.374872
1   3.390490  0.552708 -1.723864
2  -0.781308 -0.277342 -0.903127
3   0.433665 -1.125215 -0.290228
4  -2.028750 -0.083870 -0.094274
..       ...       ...       ...
95  0.443618 -1.473138  1.132161
96 -1.370215 -0.196425 -0.528401
97  1.062717 -0.997204 -1.666953
98  1.303512  0.699318 -0.863577
99 -0.109340 -1.330882 -1.455040

[100 rows x 3 columns]
Answered By: Bolster

In Jupyter, expanding on @bolster’s answer, we’ll create a reusable convenience function:

def display_n(df,n): 
    with pd.option_context('display.max_rows',n*2):
        display(df)

Then

display_n(df,2)

Returns

         0           1           2
0        0.167961    -0.732745   0.952637
1        -0.050742   -0.421239   0.444715
...      ...         ...         ...
98       0.085264    0.982093    -0.509356
99       -0.758963   -0.578267   -0.115865

(except as a nicely formatted HTML table)

when df is df = pd.DataFrame(np.random.randn(100,3))

Notes:

  1. Of course you could make the same thing print as text by modifying display to print above.
  2. On unix-like systems, you can the autoload the above function in all notebooks by placing it in a py or ipy file in ~/.ipython/profile_default/startup as described here.
Answered By: watsonic

You should use both head() and tail() for this purpose. I think the easiest way to do this is:

df.head(5).append(df.tail(5))
Answered By: Hamid

If you want to keep it to just Pandas, you can use apply() to concatenate the head and tail:

import pandas as pd
from string import ascii_lowercase, ascii_uppercase

df = pd.DataFrame(
    {"upper": list(ascii_uppercase), "lower": list(ascii_lowercase)}, index=range(1, 27)
)

df.apply(lambda x: pd.concat([x.head(2), x.tail(2)]))


   upper lower
1      A     a
2      B     b
25     Y     y
26     Z     z
Answered By: user1717828

Combining @ic_fl2 and @watsonic to give the below in Jupyter:

def ends_attr():
    def display_n(df,n):
        with pd.option_context('display.max_rows',n*2):
            display(df)
    # set pd.DataFrame attribute where .ends runs display_n() function
    setattr(pd.DataFrame,'ends',display_n)

ends_attr()

View first and last 3 rows of your df:

your_df.ends(3)

I like this because I can copy a single function and know I have everything I need to use the ends attribute.

Answered By: Me J
Categories: questions Tags: , , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.