how to ignore index comparison for pandas assert frame equal

Question

I try to compare below two dataframe with “check_index_type” set to False. According to the documentation, if it set to False, it shouldn’t “check the Index class, dtype and inferred_type are identical”. Did I misunderstood the documentation? how to compare ignoring the index and return True for below test?

I know I can reset the index but prefer not to.

https://pandas.pydata.org/pandas-docs/stable/generated/pandas.testing.assert_frame_equal.html

from pandas.util.testing import assert_frame_equal
import pandas as pd
d1 = pd.DataFrame([[1,2], [10, 20]], index=[0,2])
d2 = pd.DataFrame([[1, 2], [10, 20]], index=[0, 1])
assert_frame_equal(d1, d2, check_index_type=False)


AssertionError: DataFrame.index are different
DataFrame.index values are different (50.0 %)
[left]:  Int64Index([0, 2], dtype='int64')
[right]: Int64Index([0, 1], dtype='int64')

Asked By: Lisa

||

Source

Answer 1

Index is part of data frame , if the index are different , we should say the dataframes are different , even the value of dfs are same , so , if you want to check the value , using array_equal from numpy

d1 = pd.DataFrame([[1,2], [10, 20]], index=[0,2])
d2 = pd.DataFrame([[1, 2], [10, 20]], index=[0, 1])
np.array_equal(d1.values,d2.values)
Out[759]: True

For more info about assert_frame_equal in git

Answered By: BENY

Answer 2

If you really don’t care about the index being equal, you can drop the index as follows:

assert_frame_equal(d1.reset_index(drop=True), d2.reset_index(drop=True))

Answered By: The Aelfinn

Answer 3

For those who came to this question because they’re interested in using pd.testing.assert_series_equal (operates on pd.Series), pandas 1.1.0 has introduced an argument check_index:

import pandas as pd
s1 = pd.Series({"a": 1})
s2 = pd.Series({"b": 1})
pd.testing.assert_series_equal(s1, s2, check_index=False)

This argument does not yet exist for pd.testing.assert_frame_equals.

Answered By: OneRaynyDay

how to ignore index comparison for pandas assert frame equal

Question:

Answers: