py.test Tracebacks: Highlight my code, fold frames of framework

Question

Tracebacks of my tests are too long since I use pytest.

Pytests includes surrounding code lines and a lot of other information.

I like to see this information if the traceback line (frame) is from my code. But I don’t want to see it, if it is from a library or framework.

I could not find a way to filter or fold the frames.

Any hints?

Update, 8 years later: I think it is time to say goodbye to ASCII and embrace html. With html you can expand/collapse parts (like in the great django debug view).

Unfortunately there seems to be no pytest output which gives you nice interface like for example sentry does.

Asked By: guettli

||

Source

Answer 1

I’m afraid this is currently not possible and is an enhancement request: https://bitbucket.org/hpk42/pytest/issue/283/traceback-filtering-hook

The only control you have for the moment is --tb=short etc.

Answered By: flub

Answer 2

I’ve got this monkeypatch I’m sticking in my conftest.py
when I want to do this (only works with --tb=native):

It’s a port of this https://stackoverflow.com/a/24679193/59412 by Jeremy Allen to the latest version of pytest, 6.2.3. It’s certainly not the cleanest thing, but it removes all code from 3rd party dependencies
(e.g. anything pip installed) from the tracebacks.

# 5c4048fc-ccf1-44ab-a683-78a29c1a98a6
import _pytest._code.code
def should_skip_line(line):
    """
    decide which lines to skip
    """
    return 'site-packages' in line

class PatchedReprEntryNative(_pytest._code.code.ReprEntryNative):
    def __init__(self, tblines):
        self.lines = []
        while len(tblines) > 0:
            # [...yourfilter...]/framework_code.py", line 1, in test_thing
            line = tblines.pop(0)
            if should_skip_line(line):
                # the line of framework code you don't want to see either...
                tblines.pop(0)
            else:
                self.lines.append(line)
_pytest._code.code.ReprEntryNative = PatchedReprEntryNative
del _pytest._code.code

Answered By: neozen

Answer 3

Another option that monkey patch pytest, but works without the --tb=native option. Just put this code in your contest.py:

import pathlib
from _pytest._code import code
from _pytest._code.code import ReprTraceback

def ishidden(self) -> bool:
    return self._ishidden() or 'site-packages' in pathlib.Path(self.path).parts
code.TracebackEntry._ishidden = code.TracebackEntry.ishidden
code.TracebackEntry.ishidden = ishidden

def repr_traceback(self, excinfo: code.ExceptionInfo[BaseException]) -> "ReprTraceback":
    traceback = excinfo.traceback
    if True:  # self.tbfilter:  <- filtering was not done for nested exception, so force it
        traceback = traceback.filter()
    # make sure we don't get an empty traceback list
    if len(traceback) == 0:
        traceback.append(excinfo.traceback[-1])

    if isinstance(excinfo.value, RecursionError):
        traceback, extraline = self._truncate_recursive_traceback(traceback)
    else:
        extraline = None

    last = traceback[-1]
    entries = []
    if self.style == "value":
        reprentry = self.repr_traceback_entry(last, excinfo)
        entries.append(reprentry)
        return ReprTraceback(entries, None, style=self.style)

    for index, entry in enumerate(traceback):
        einfo = (last == entry) and excinfo or None
        reprentry = self.repr_traceback_entry(entry, einfo)
        entries.append(reprentry)
    return ReprTraceback(entries, extraline, style=self.style)
code.FormattedExcinfo.repr_traceback = repr_traceback

del code

We get a much better output when using external libraries like playwright, making the default traceback format actually usable:

__________________________________________________________________ test_authenticated_access_to_the_application ___________________________________________________________________

page = <Page url='about:blank'>, base_url = ''

    @when('I go to the application')
    def provide_creds(page: Page, base_url: str):
        with page.expect_navigation(wait_until='networkidle', timeout=5000):
>           page.goto(base_url)
E           playwright._impl._api_types.Error: Protocol error (Page.navigate): Cannot navigate to invalid URL
E           =========================== logs ===========================
E           navigating to "", waiting until "load"
E           ============================================================

tests/gui/step_defs/star_steps.py:16: Error

During handling of the above exception, another exception occurred:

page = <Page url='about:blank'>, base_url = ''

    @when('I go to the application')
    def provide_creds(page: Page, base_url: str):
>       with page.expect_navigation(wait_until='networkidle', timeout=5000):
E       playwright._impl._api_types.TimeoutError: Timeout 5000ms exceeded.
E       =========================== logs ===========================
E       waiting for navigation until 'networkidle'
E       ============================================================

tests/gui/step_defs/star_steps.py:15: TimeoutError

compared to the unpatched version:

__________________________________________________________________ test_authenticated_access_to_the_application ___________________________________________________________________

page = <Page url='about:blank'>, base_url = ''

    @when('I go to the application')
    def provide_creds(page: Page, base_url: str):
        with page.expect_navigation(wait_until='networkidle', timeout=5000):
>           page.goto(base_url)

tests/gui/step_defs/star_steps.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Page url='about:blank'>, url = ''

    def goto(
        self,
        url: str,
        *,
        timeout: float = None,
        wait_until: Literal["commit", "domcontentloaded", "load", "networkidle"] = None,
        referer: str = None
    ) -> typing.Optional["Response"]:
        """Page.goto
    
        Returns the main resource response. In case of multiple redirects, the navigation will resolve with the first
        non-redirect response.
    
        The method will throw an error if:
        - there's an SSL error (e.g. in case of self-signed certificates).
        - target URL is invalid.
        - the `timeout` is exceeded during navigation.
        - the remote server does not respond or is unreachable.
        - the main resource failed to load.
    
        The method will not throw an error when any valid HTTP status code is returned by the remote server, including 404 "Not
        Found" and 500 "Internal Server Error".  The status code for such responses can be retrieved by calling
        `response.status()`.
    
        > NOTE: The method either throws an error or returns a main resource response. The only exceptions are navigation to
        `about:blank` or navigation to the same URL with a different hash, which would succeed and return `null`.
        > NOTE: Headless mode doesn't support navigation to a PDF document. See the
        [upstream issue](https://bugs.chromium.org/p/chromium/issues/detail?id=761295).
    
        Shortcut for main frame's `frame.goto()`
    
        Parameters
        ----------
        url : str
            URL to navigate page to. The url should include scheme, e.g. `https://`. When a `baseURL` via the context options was
            provided and the passed URL is a path, it gets merged via the
            [`new URL()`](https://developer.mozilla.org/en-US/docs/Web/API/URL/URL) constructor.
        timeout : Union[float, NoneType]
            Maximum operation time in milliseconds, defaults to 30 seconds, pass `0` to disable timeout. The default value can be
            changed by using the `browser_context.set_default_navigation_timeout()`,
            `browser_context.set_default_timeout()`, `page.set_default_navigation_timeout()` or
            `page.set_default_timeout()` methods.
        wait_until : Union["commit", "domcontentloaded", "load", "networkidle", NoneType]
            When to consider operation succeeded, defaults to `load`. Events can be either:
            - `'domcontentloaded'` - consider operation to be finished when the `DOMContentLoaded` event is fired.
            - `'load'` - consider operation to be finished when the `load` event is fired.
            - `'networkidle'` - consider operation to be finished when there are no network connections for at least `500` ms.
            - `'commit'` - consider operation to be finished when network response is received and the document started loading.
        referer : Union[str, NoneType]
            Referer header value. If provided it will take preference over the referer header value set by
            `page.set_extra_http_headers()`.
    
        Returns
        -------
        Union[Response, NoneType]
        """
    
        return mapping.from_impl_nullable(
>           self._sync(
                self._impl_obj.goto(
                    url=url, timeout=timeout, waitUntil=wait_until, referer=referer
                )
            )
        )

.venv/lib/python3.10/site-packages/playwright/sync_api/_generated.py:7285: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Page url='about:blank'>, coro = <coroutine object Page.goto at 0x7f3b6b5e1700>

    def _sync(self, coro: Awaitable) -> Any:
        __tracebackhide__ = True
        g_self = greenlet.getcurrent()
        task = self._loop.create_task(coro)
        setattr(task, "__pw_stack__", inspect.stack())
        setattr(task, "__pw_stack_trace__", traceback.extract_stack())
    
        task.add_done_callback(lambda _: g_self.switch())
        while not task.done():
            self._dispatcher_fiber.switch()
        asyncio._set_running_loop(self._loop)
>       return task.result()

.venv/lib/python3.10/site-packages/playwright/_impl/_sync_base.py:89: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Page url='about:blank'>, url = '', timeout = None, waitUntil = None, referer = None

    async def goto(
        self,
        url: str,
        timeout: float = None,
        waitUntil: DocumentLoadState = None,
        referer: str = None,
    ) -> Optional[Response]:
>       return await self._main_frame.goto(**locals_to_params(locals()))

.venv/lib/python3.10/site-packages/playwright/_impl/_page.py:496: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Frame name= url='about:blank'>, url = '', timeout = None, waitUntil = None, referer = None

    async def goto(
        self,
        url: str,
        timeout: float = None,
        waitUntil: DocumentLoadState = None,
        referer: str = None,
    ) -> Optional[Response]:
        return cast(
            Optional[Response],
            from_nullable_channel(
>               await self._channel.send("goto", locals_to_params(locals()))
            ),
        )

.venv/lib/python3.10/site-packages/playwright/_impl/_frame.py:136: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <playwright._impl._connection.Channel object at 0x7f3b7095e7a0>, method = 'goto', params = {'url': ''}

    async def send(self, method: str, params: Dict = None) -> Any:
>       return await self._connection.wrap_api_call(
            lambda: self.inner_send(method, params, False)
        )

.venv/lib/python3.10/site-packages/playwright/_impl/_connection.py:43: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <playwright._impl._connection.Connection object at 0x7f3b70976cb0>, cb = <function Channel.send.<locals>.<lambda> at 0x7f3b6b5a53f0>, is_internal = False

    async def wrap_api_call(
        self, cb: Callable[[], Any], is_internal: bool = False
    ) -> Any:
        if self._api_zone.get():
            return await cb()
        task = asyncio.current_task(self._loop)
        st: List[inspect.FrameInfo] = getattr(task, "__pw_stack__", inspect.stack())
        metadata = _extract_metadata_from_stack(st, is_internal)
        if metadata:
            self._api_zone.set(metadata)
        try:
>           return await cb()

.venv/lib/python3.10/site-packages/playwright/_impl/_connection.py:369: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <playwright._impl._connection.Channel object at 0x7f3b7095e7a0>, method = 'goto', params = {'url': ''}, return_as_dict = False

    async def inner_send(
        self, method: str, params: Optional[Dict], return_as_dict: bool
    ) -> Any:
        if params is None:
            params = {}
        callback = self._connection._send_message_to_server(self._guid, method, params)
        if self._connection._error:
            error = self._connection._error
            self._connection._error = None
            raise error
        done, _ = await asyncio.wait(
            {
                self._connection._transport.on_error_future,
                callback.future,
            },
            return_when=asyncio.FIRST_COMPLETED,
        )
        if not callback.future.done():
            callback.future.cancel()
>       result = next(iter(done)).result()
E       playwright._impl._api_types.Error: Protocol error (Page.navigate): Cannot navigate to invalid URL
E       =========================== logs ===========================
E       navigating to "", waiting until "load"
E       ============================================================

.venv/lib/python3.10/site-packages/playwright/_impl/_connection.py:78: Error

During handling of the above exception, another exception occurred:

page = <Page url='about:blank'>, base_url = ''

    @when('I go to the application')
    def provide_creds(page: Page, base_url: str):
>       with page.expect_navigation(wait_until='networkidle', timeout=5000):

tests/gui/step_defs/star_steps.py:15: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv/lib/python3.10/site-packages/playwright/_impl/_sync_base.py:66: in __exit__
    self._event.value
.venv/lib/python3.10/site-packages/playwright/_impl/_sync_base.py:46: in value
    raise exception
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    async def continuation() -> Optional[Response]:
>       event = await wait_helper.result()
E       playwright._impl._api_types.TimeoutError: Timeout 5000ms exceeded.
E       =========================== logs ===========================
E       waiting for navigation until 'networkidle'
E       ============================================================

.venv/lib/python3.10/site-packages/playwright/_impl/_frame.py:197: TimeoutError

Answered By: Gaëtan Lehmann

py.test Tracebacks: Highlight my code, fold frames of framework

Question:

Answers: