Report pytest-stress results individually for each iteration in pytest-html

Question:

I need to use pytest-stress to run tests for a period of time. Previously, I was using pytest-repeat which is based on the number of iterations. I am using pytest-html for reporting results.

When I switched to stress from repeat, all iterations of the test would all appear inside 1 test row. Repeat was able to separate the tests into different iterations.
I am trying to get stress to just list the test and the current iteration count in a separate row for pytest-html.

I know that pytest-stress is overriding pytest_runtestloop so it should be running at the session scope level.
I did try adding in some of the functionality from pytest_generate_tests that pytest-repeat overrides because it runs at the function scope level.

I would like to for the results to report each iteration separately (I removed the preceding path for readability here)

ex:

test_report.py::TestReport::test_fail[1]

test_report.py::TestReport::test_fail[2]

test_report.py::TestReport::test_fail[3]

pytest-repeat.png
pytest-stress.png

Example Code:

import logging
class TestReport:
    def test_pass(self):
        logging.info("assert(True)")
        assert(True)

    def test_fail(self):
        logging.info("assert(False)")
        assert(False)

Conftest.py

def pytest_html_results_table_header(cells):
    cells.insert(2, html.th("Time", class_="sortable time", col="time"))
    cells.pop()


def pytest_html_results_table_row(report, cells):
    cells.insert(2, html.td(datetime.utcnow(), class_="col-time"))
    cells.pop()


@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    report = outcome.get_result()
    setattr(report, "duration_formatter", "%H:%M:%S.%f")

@pytest.hookimpl(tryfirst=True)
def pytest_configure(config):
    if not os.path.exists('results'):
        os.makedirs('results')
    config.option.htmlpath = 'results/results_'+datetime.now().strftime("%Y-%m-%d_T%H-%M-%S")+".html"

#change name based on iteration
def pytest_itemcollected(item):
    cls = item.getparent(pytest.Class)
    iter = 0
    if hasattr(cls.obj, "iteration"):
        iter = int(getattr(cls.obj, "iteration"))
    item._nodeid = item._nodeid + f"[{iter}]"


I do not know if this is possible through a minor edit to conftest.py or if I would have to create my own plugin for pytest to run based on a period of time.

Asked By: anogowski

||

Answers:

Created my own pytest plugin called pytest-loop. It merges pytest-stress and pytest-repeat with a fix to clear previous reports.

https://github.com/anogowski/pytest-loop

https://pypi.org/project/pytest-loop/

Answered By: anogowski
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.