How can I limit the maximum running time for a unit test?

Question

I am currently running some unit tests that might either take a long time before failing or run indefinitely. In a successful test run they will always complete within a certain amount of time.

Is it possible to create a pytest unit test that will fail if it does not complete within a certain amount of time?

Asked By: shuttle87

||

Source

Answer 1

you can install the pytest-timeout plugin and then mark your test functions with a timeout in seconds.

@pytest.mark.timeout(300)
def test_foo():
   pass

Look at the plugin download and usage instructions at https://pypi.python.org/pypi/pytest-timeout

Answered By: Mrinal Shukla

Answer 2

Bash functionality can be used:

EXIT_CODE=0
TIME_LIMIT=60

timeout $TIME_LIMIT pytest ... || EXIT_CODE=$?

if [ $EXIT_CODE -ne 0 ]; then
    
    echo "Your error message to log"
    
    ...
    
    exit $EXIT_CODE
    
fi

Answered By: Konstantin

Answer 3

Here is a lightweight way that you can use in a doctest or directly in your script and without any extra dependency, inspired from a question on how to limit execution time for a function

import signal
from contextlib import contextmanager


@contextmanager
def time_limit(seconds):
    def signal_handler(signum, frame):
        raise TimeoutError
    signal.signal(signal.SIGALRM, signal_handler)
    signal.alarm(seconds)
    try:
        yield
    finally:
        signal.alarm(0)

def efficiency_test(function, *args, **kwargs):
    try:
        with time_limit(2):
            return (True, function(*args, **kwargs))
    except TimeoutError as e:
        return (False, None)

With example usage:

import signal
from contextlib import contextmanager


@contextmanager
def time_limit(seconds):
    def signal_handler(signum, frame):
        raise TimeoutError
    signal.signal(signal.SIGALRM, signal_handler)
    signal.alarm(seconds)
    try:
        yield
    finally:
        signal.alarm(0)

def simple(a, b):
    return a + b

def complex_(a, b):
    for i in range(100000000):
        a += b
    return a

def efficiency_test(function, *args, **kwargs):
    try:
        with time_limit(2):
            return (True, function(*args, **kwargs))
    except TimeoutError as e:
        return (False, None)

if __name__ == "__main__":
    print(efficiency_test(simple, 1, 2))
    print(efficiency_test(complex_, 1, 2))

With output:

(True, 3)
(False, None)

Answered By: Caridorc

How can I limit the maximum running time for a unit test?

Question:

Answers: