Pytest test fails for all cases when only one fails

Question

I have a Pytest file structured as below:


@pytest.fixture(scope="class")
def driver_init(request):
    driver = webdriver.Chrome("path tp driver")
    driver.maximize_window()
    request.cls.driver = driver
    driver.get(url)
    # code that navigates to a page that I need to validate data on 
    yield
    driver.close()
    
pytest.mark.usefixtures("driver_init")
@pytest.mark.parametrize("user", get_users()) 
class TestValidate:
    @pytest.mark.order(1)
    def test_nav(self, user):
        username = user[0]
        # code that updates page to get data that depends on username

    def test_data1(self, user):
        data_1 = user[1]
        assert driver.find_element("id", "location_of_data1").text == data_1
        
    def test_data2(self, user):
        data_2 = user[2]
        assert driver.find_element("id", "location_of_data2").text == data_2
    
    def test_data3(self, user):
        data_3 = user[3]
        assert driver.find_element("id", "location_of_data3").text == data_3

get_users() returns a list of lists, e.g. [["Username1", "data1", "data2", "data3"],["Username2", "data1", "data2", "data3"]]

The tests seem to function correctly, and it works perfectly when there is only 1 set of test data (e.g., only Username1 and its corresponding data). However, when there is 2 sets of test data and one of the tests fails, Pytest reports that test_data3 failed for both sets of data, even though it passed for one of them

Example: if the expected results of the two sets are [Pass, Pass, Pass] and [Pass, Pass, Fail], Pytest will fail 2 out of 6 tests even though only 1 of 6 failed.

To see what would happen, I added a third set of test data (which should be [Pass, Pass, Pass]). When this test case is the last one to be executed, Pytest reports that all the tests passed, even though the third test using the second set of test data did not pass.

Any help appreciated even if I have to completely restructure things

Asked By: mbeansly

||

Source

Answer 1

Your fixture is declared to be scope="class", so browser state is going to leak between your unit tests.

Remove it, and the default scope should make each unit test independent by using a dedicated browser instance per test.

Otherwise, you’ll need to ensure that the browser state is "reset" as much as necessary for each test.

Answered By: Kache

Pytest test fails for all cases when only one fails

Question:

Answers: