select pandas rows by excluding index number

Question

Not quite sure why I can’t figure this out. I’m looking to slice a Pandas dataframe by using index numbers. I have a list/core index with the index numbers that i do NOT need, shown below

 pandas.core.index.Int64Index

 Int64Index([2340, 4840, 3163, 1597, 491 , 5010, 911 , 3085, 5486, 5475, 1417, 2663, 4204, 156 , 5058, 1990, 3200, 1218, 3280, 793 , 824 , 3625, 1726, 1971, 2845, 4668, 2973, 3039, 376 , 4394, 3749, 1610, 3892, 2527, 324 , 5245, 696 , 1239, 4601, 3219, 5138, 4832, 4762, 1256, 4437, 2475, 3732, 4063, 1193], dtype=int64)

How can I create a new dataframe excluding these index numbers. I tried

df.iloc[combined_index]

and obviously this just shows the rows with those index number (the opposite of what I want). any help will be greatly appreciated

Asked By: itjcms18

||

Source

Answer 1

You could use pd.Int64Index(np.arange(len(df))).difference(index) to form a new ordinal index. For example, if we want to remove the rows associated with ordinal index [1,3,5], then

import numpy as np
import pandas as pd

index = pd.Int64Index([1,3,5], dtype=np.int64)
df = pd.DataFrame(np.arange(6*2).reshape((6,2)), index=list('ABCDEF'))
#     0   1
# A   0   1
# B   2   3
# C   4   5
# D   6   7
# E   8   9
# F  10  11

new_index = pd.Int64Index(np.arange(len(df))).difference(index)
print(df.iloc[new_index])

yields

Answered By: unutbu

Answer 2

Not sure if that’s what you are looking for, posting this as an answer, because it’s too long for a comment:

In [31]: d = {'a':[1,2,3,4,5,6], 'b':[1,2,3,4,5,6]}

In [32]: df = pd.DataFrame(d)

In [33]: bad_df = df.index.isin([3,5])

In [34]: df[~bad_df]
Out[34]: 
   a  b
0  1  1
1  2  2
2  3  3
4  5  5

Answered By: Vor

Answer 3

Probably an easier way is just to use a boolean index, and slice normally doing something like this:

df[~df.index.isin(list_to_exclude)]

Answered By: Allen Wang

Answer 4

Just use .drop and pass it the index list to exclude.

import pandas as pd

df = pd.DataFrame({"a": [10, 11, 12, 13, 14, 15]})


df.drop([1, 2, 3], axis=0)

Which outputs this.

Answered By: Chris Farr

Answer 5

Assuming there exists a DataFrame df:

In [4]: df = pd.DataFrame({'a': range(4), 'b': ['a', 'b', 'c', 'd']})

In [5]: df
Out[5]: 
   a  b
0  0  a
1  1  b
2  2  c
3  3  d

and you want to remove index [1, 3], you can use query:

In [5]: df.query('index != [1,3]')
Out[5]: 
   a  b
0  0  a
2  2  c

Answered By: rachwa

select pandas rows by excluding index number

Question:

Answers: