Logging within pytest tests
Question:
I would like to put some logging statements within test function to examine some state variables.
I have the following code snippet:
import pytest,os
import logging
logging.basicConfig(level=logging.DEBUG)
mylogger = logging.getLogger()
#############################################################################
def setup_module(module):
''' Setup for the entire module '''
mylogger.info('Inside Setup')
# Do the actual setup stuff here
pass
def setup_function(func):
''' Setup for test functions '''
if func == test_one:
mylogger.info(' Hurray !!')
def test_one():
''' Test One '''
mylogger.info('Inside Test 1')
#assert 0 == 1
pass
def test_two():
''' Test Two '''
mylogger.info('Inside Test 2')
pass
if __name__ == '__main__':
mylogger.info(' About to start the tests ')
pytest.main(args=[os.path.abspath(__file__)])
mylogger.info(' Done executing the tests ')
I get the following output:
[bmaryada-mbp:/Users/bmaryada/dev/platform/main/proto/tests/tpch $]python minitest.py
INFO:root: About to start the tests
======================================================== test session starts =========================================================
platform darwin -- Python 2.6.2 -- pytest-2.0.0
collected 2 items
minitest.py ..
====================================================== 2 passed in 0.01 seconds ======================================================
INFO:root: Done executing the tests
Notice that only the logging messages from the '__name__ == __main__' block get transmitted to the console.
Is there a way to force pytest to emit logging to console from test methods as well?
Answers:
Works for me, here’s the output I get: [snip -> example was incorrect]
Edit: It seems that you have to pass the -s option to py.test so it won’t capture stdout. Here (py.test not installed), it was enough to use python pytest.py -s pyt.py.
For your code, all you need is to pass -s in args to main:
pytest.main(args=['-s', os.path.abspath(__file__)])
See the py.test documentation on capturing output.
Since version 3.3, pytest supports live logging, meaning that all the log records emitted in tests will be printed to the terminal immediately. The feature is documented under Live Logs section. Live logging is disabled by default; to enable it, set log_cli = 1 in the pyproject.toml1 or pytest.ini2 config. Live logging supports emitting to terminal and file; the relevant options allow records customizing:
terminal:
log_cli_levellog_cli_formatlog_cli_date_format
file:
log_filelog_file_levellog_file_formatlog_file_date_format
As pointed out by Kévin Barré in this comment, overriding ini options from command line can be done via:
-o OVERRIDE_INI, --override-ini=OVERRIDE_INI
override ini option with "option=value" style, e.g.
-o xfail_strict=True -o cache_dir=cache
So instead of declaring log_cli in pytest.ini, you can simply call:
$ pytest -o log_cli=true ...
Examples
Simple test file used for demonstrating:
# test_spam.py
import logging
LOGGER = logging.getLogger(__name__)
def test_eggs():
LOGGER.info('eggs info')
LOGGER.warning('eggs warning')
LOGGER.error('eggs error')
LOGGER.critical('eggs critical')
assert True
As you can see, no extra configuration needed; pytest will setup the logger automatically, based on options specified in pytest.ini or passed from command line.
Live logging to terminal, INFO level, fancy output
Configuration in pyproject.toml:
[tool.pytest.ini_options]
log_cli = true
log_cli_level = "INFO"
log_cli_format = "%(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)"
log_cli_date_format = "%Y-%m-%d %H:%M:%S"
The identical configuration in legacy pytest.ini:
[pytest]
log_cli = 1
log_cli_level = INFO
log_cli_format = %(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)
log_cli_date_format=%Y-%m-%d %H:%M:%S
Running the test:
$ pytest test_spam.py
=============================== test session starts ================================
platform darwin -- Python 3.6.4, pytest-3.7.0, py-1.5.3, pluggy-0.7.1 -- /Users/hoefling/.virtualenvs/stackoverflow/bin/python3.6
cachedir: .pytest_cache
rootdir: /Users/hoefling/projects/private/stackoverflow/so-4673373, inifile: pytest.ini
collected 1 item
test_spam.py::test_eggs
---------------------------------- live log call -----------------------------------
2018-08-01 14:33:20 [ INFO] eggs info (test_spam.py:7)
2018-08-01 14:33:20 [ WARNING] eggs warning (test_spam.py:8)
2018-08-01 14:33:20 [ ERROR] eggs error (test_spam.py:9)
2018-08-01 14:33:20 [CRITICAL] eggs critical (test_spam.py:10)
PASSED [100%]
============================= 1 passed in 0.01 seconds =============================
Live logging to terminal and file, only message & CRITICAL level in terminal, fancy output in pytest.log file
Configuration in pyproject.toml:
[tool.pytest.ini_options]
log_cli = true
log_cli_level = "CRITICAL"
log_cli_format = "%(message)s"
log_file = "pytest.log"
log_file_level = "DEBUG"
log_file_format = "%(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)"
log_file_date_format = "%Y-%m-%d %H:%M:%S"
The identical configuration in legacy pytest.ini:
[pytest]
log_cli = 1
log_cli_level = CRITICAL
log_cli_format = %(message)s
log_file = pytest.log
log_file_level = DEBUG
log_file_format = %(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)
log_file_date_format=%Y-%m-%d %H:%M:%S
Test run:
$ pytest test_spam.py
=============================== test session starts ================================
platform darwin -- Python 3.6.4, pytest-3.7.0, py-1.5.3, pluggy-0.7.1 -- /Users/hoefling/.virtualenvs/stackoverflow/bin/python3.6
cachedir: .pytest_cache
rootdir: /Users/hoefling/projects/private/stackoverflow/so-4673373, inifile: pytest.ini
collected 1 item
test_spam.py::test_eggs
---------------------------------- live log call -----------------------------------
eggs critical
PASSED [100%]
============================= 1 passed in 0.01 seconds =============================
$ cat pytest.log
2018-08-01 14:38:09 [ INFO] eggs info (test_spam.py:7)
2018-08-01 14:38:09 [ WARNING] eggs warning (test_spam.py:8)
2018-08-01 14:38:09 [ ERROR] eggs error (test_spam.py:9)
2018-08-01 14:38:09 [CRITICAL] eggs critical (test_spam.py:10)
1 pyproject.toml supported since version 6.0 and is the best option IMO. See PEP 518 for the specs.
2 Although you can also configure pytest in setup.cfg under the [tool:pytest] section, don’t be tempted to do that when you want to provide custom live logging format. Other tools reading setup.cfg might treat stuff like %(message)s as string interpolation and fail. The best choice is using pyproject.toml anyway, but if you are forced to use the legacy ini-style format, stick to pytest.ini to avoid errors.
Using pytest --log-cli-level=DEBUG works fine with pytest (tested from 6.2.2 to 7.1.1)
Using pytest --log-cli-level=DEBUG --capture=tee-sys will also print stdtout.
To turn logger output on use send --capture=no flag from command line.
--capture=no will show all outputs from logger and print statements. If you would like to capture outputs from logger and not print statements use --capture=sys
pytest --capture=no tests/system/test_backoffice.py
Here is more information about "Capturing of the stdout/stderr output"
By default logger output level is "WARNING"
To change log output level use --log-cli-level flag.
pytest --capture=no --log-cli-level=DEBUG tests/system/test_backoffice.py
If you use vscode, use following config, assuming you’ve installed
Python official plugin (ms-python.python) for your python project.
./.vscode/setting.json under your proj
{
....
"python.testing.pytestArgs": ["-s", "src"], //here before discover-path src
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pytestEnabled": true,
...
}
P.S. Some plugins work on it, including but not limited to:
- Python Test Explorer for Visual Studio Code (
littlefoxteam.vscode-python-test-adapter)
- Test Explorer for Visual Studio Code(
hbenl.vscode-test-explorer)
You can read:
Documentation for logging in pytest
Here is simple example that you can run and get log from foo function.
#./test_main.py
from main import foo
import logging
def test_foo(caplog):
caplog.set_level(logging.INFO)
logging.getLogger().info('Log inside a test function!')
assert foo(1, 2) == 5
/* your test here*/
# ./main.py
import logging
def foo(a, b):
logging.getLogger().info('a: ' + str(a))
logging.getLogger().info('b: ' + str(b))
return a + b
Now you can run pytest and get log info from function that you need.
If you don’t have any errors, logs will be omitted.
If you want to filter logs with the command line, you can pass –log-cli-level (pytest –log-cli-level)
and logs will be shown from the level you specified and above
(e.g. pytest –log-cli-level=INFO will show INFO and above logs(WARNING, ERROR, CRITICAL))
note that: default –log-cli-level is a WARNING if you don’t specify it (https://docs.pytest.org/en/6.2.x/logging.html)
But if you dont want to use –log-cli-level every time use pytest,
you can set log-level
in your pytest config file (pytest.ini/tox.ini/setup.cfg)
e.g.
put log-level=INFO in pytest.ini (or other config files i mentioned)
when you run pytest, you only see INFO and above logs
I would like to put some logging statements within test function to examine some state variables.
I have the following code snippet:
import pytest,os
import logging
logging.basicConfig(level=logging.DEBUG)
mylogger = logging.getLogger()
#############################################################################
def setup_module(module):
''' Setup for the entire module '''
mylogger.info('Inside Setup')
# Do the actual setup stuff here
pass
def setup_function(func):
''' Setup for test functions '''
if func == test_one:
mylogger.info(' Hurray !!')
def test_one():
''' Test One '''
mylogger.info('Inside Test 1')
#assert 0 == 1
pass
def test_two():
''' Test Two '''
mylogger.info('Inside Test 2')
pass
if __name__ == '__main__':
mylogger.info(' About to start the tests ')
pytest.main(args=[os.path.abspath(__file__)])
mylogger.info(' Done executing the tests ')
I get the following output:
[bmaryada-mbp:/Users/bmaryada/dev/platform/main/proto/tests/tpch $]python minitest.py
INFO:root: About to start the tests
======================================================== test session starts =========================================================
platform darwin -- Python 2.6.2 -- pytest-2.0.0
collected 2 items
minitest.py ..
====================================================== 2 passed in 0.01 seconds ======================================================
INFO:root: Done executing the tests
Notice that only the logging messages from the '__name__ == __main__' block get transmitted to the console.
Is there a way to force pytest to emit logging to console from test methods as well?
Works for me, here’s the output I get: [snip -> example was incorrect]
Edit: It seems that you have to pass the -s option to py.test so it won’t capture stdout. Here (py.test not installed), it was enough to use python pytest.py -s pyt.py.
For your code, all you need is to pass -s in args to main:
pytest.main(args=['-s', os.path.abspath(__file__)])
See the py.test documentation on capturing output.
Since version 3.3, pytest supports live logging, meaning that all the log records emitted in tests will be printed to the terminal immediately. The feature is documented under Live Logs section. Live logging is disabled by default; to enable it, set log_cli = 1 in the pyproject.toml1 or pytest.ini2 config. Live logging supports emitting to terminal and file; the relevant options allow records customizing:
terminal:
log_cli_levellog_cli_formatlog_cli_date_format
file:
log_filelog_file_levellog_file_formatlog_file_date_format
As pointed out by Kévin Barré in this comment, overriding ini options from command line can be done via:
-o OVERRIDE_INI, --override-ini=OVERRIDE_INI
override ini option with "option=value" style, e.g.
-o xfail_strict=True -o cache_dir=cache
So instead of declaring log_cli in pytest.ini, you can simply call:
$ pytest -o log_cli=true ...
Examples
Simple test file used for demonstrating:
# test_spam.py
import logging
LOGGER = logging.getLogger(__name__)
def test_eggs():
LOGGER.info('eggs info')
LOGGER.warning('eggs warning')
LOGGER.error('eggs error')
LOGGER.critical('eggs critical')
assert True
As you can see, no extra configuration needed; pytest will setup the logger automatically, based on options specified in pytest.ini or passed from command line.
Live logging to terminal, INFO level, fancy output
Configuration in pyproject.toml:
[tool.pytest.ini_options]
log_cli = true
log_cli_level = "INFO"
log_cli_format = "%(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)"
log_cli_date_format = "%Y-%m-%d %H:%M:%S"
The identical configuration in legacy pytest.ini:
[pytest]
log_cli = 1
log_cli_level = INFO
log_cli_format = %(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)
log_cli_date_format=%Y-%m-%d %H:%M:%S
Running the test:
$ pytest test_spam.py
=============================== test session starts ================================
platform darwin -- Python 3.6.4, pytest-3.7.0, py-1.5.3, pluggy-0.7.1 -- /Users/hoefling/.virtualenvs/stackoverflow/bin/python3.6
cachedir: .pytest_cache
rootdir: /Users/hoefling/projects/private/stackoverflow/so-4673373, inifile: pytest.ini
collected 1 item
test_spam.py::test_eggs
---------------------------------- live log call -----------------------------------
2018-08-01 14:33:20 [ INFO] eggs info (test_spam.py:7)
2018-08-01 14:33:20 [ WARNING] eggs warning (test_spam.py:8)
2018-08-01 14:33:20 [ ERROR] eggs error (test_spam.py:9)
2018-08-01 14:33:20 [CRITICAL] eggs critical (test_spam.py:10)
PASSED [100%]
============================= 1 passed in 0.01 seconds =============================
Live logging to terminal and file, only message & CRITICAL level in terminal, fancy output in pytest.log file
Configuration in pyproject.toml:
[tool.pytest.ini_options]
log_cli = true
log_cli_level = "CRITICAL"
log_cli_format = "%(message)s"
log_file = "pytest.log"
log_file_level = "DEBUG"
log_file_format = "%(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)"
log_file_date_format = "%Y-%m-%d %H:%M:%S"
The identical configuration in legacy pytest.ini:
[pytest]
log_cli = 1
log_cli_level = CRITICAL
log_cli_format = %(message)s
log_file = pytest.log
log_file_level = DEBUG
log_file_format = %(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)
log_file_date_format=%Y-%m-%d %H:%M:%S
Test run:
$ pytest test_spam.py
=============================== test session starts ================================
platform darwin -- Python 3.6.4, pytest-3.7.0, py-1.5.3, pluggy-0.7.1 -- /Users/hoefling/.virtualenvs/stackoverflow/bin/python3.6
cachedir: .pytest_cache
rootdir: /Users/hoefling/projects/private/stackoverflow/so-4673373, inifile: pytest.ini
collected 1 item
test_spam.py::test_eggs
---------------------------------- live log call -----------------------------------
eggs critical
PASSED [100%]
============================= 1 passed in 0.01 seconds =============================
$ cat pytest.log
2018-08-01 14:38:09 [ INFO] eggs info (test_spam.py:7)
2018-08-01 14:38:09 [ WARNING] eggs warning (test_spam.py:8)
2018-08-01 14:38:09 [ ERROR] eggs error (test_spam.py:9)
2018-08-01 14:38:09 [CRITICAL] eggs critical (test_spam.py:10)
1 pyproject.toml supported since version 6.0 and is the best option IMO. See PEP 518 for the specs.
2 Although you can also configure pytest in setup.cfg under the [tool:pytest] section, don’t be tempted to do that when you want to provide custom live logging format. Other tools reading setup.cfg might treat stuff like %(message)s as string interpolation and fail. The best choice is using pyproject.toml anyway, but if you are forced to use the legacy ini-style format, stick to pytest.ini to avoid errors.
Using pytest --log-cli-level=DEBUG works fine with pytest (tested from 6.2.2 to 7.1.1)
Using pytest --log-cli-level=DEBUG --capture=tee-sys will also print stdtout.
To turn logger output on use send --capture=no flag from command line.
--capture=no will show all outputs from logger and print statements. If you would like to capture outputs from logger and not print statements use --capture=sys
pytest --capture=no tests/system/test_backoffice.py
Here is more information about "Capturing of the stdout/stderr output"
By default logger output level is "WARNING"
To change log output level use --log-cli-level flag.
pytest --capture=no --log-cli-level=DEBUG tests/system/test_backoffice.py
If you use vscode, use following config, assuming you’ve installed
Python official plugin (ms-python.python) for your python project.
./.vscode/setting.json under your proj
{
....
"python.testing.pytestArgs": ["-s", "src"], //here before discover-path src
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pytestEnabled": true,
...
}
P.S. Some plugins work on it, including but not limited to:
- Python Test Explorer for Visual Studio Code (
littlefoxteam.vscode-python-test-adapter) - Test Explorer for Visual Studio Code(
hbenl.vscode-test-explorer)
You can read:
Documentation for logging in pytest
Here is simple example that you can run and get log from foo function.
#./test_main.py
from main import foo
import logging
def test_foo(caplog):
caplog.set_level(logging.INFO)
logging.getLogger().info('Log inside a test function!')
assert foo(1, 2) == 5
/* your test here*/
# ./main.py
import logging
def foo(a, b):
logging.getLogger().info('a: ' + str(a))
logging.getLogger().info('b: ' + str(b))
return a + b
Now you can run pytest and get log info from function that you need.
If you don’t have any errors, logs will be omitted.
If you want to filter logs with the command line, you can pass –log-cli-level (pytest –log-cli-level)
and logs will be shown from the level you specified and above
(e.g. pytest –log-cli-level=INFO will show INFO and above logs(WARNING, ERROR, CRITICAL))
note that: default –log-cli-level is a WARNING if you don’t specify it (https://docs.pytest.org/en/6.2.x/logging.html)
But if you dont want to use –log-cli-level every time use pytest,
you can set log-level
in your pytest config file (pytest.ini/tox.ini/setup.cfg)
e.g.
put log-level=INFO in pytest.ini (or other config files i mentioned)
when you run pytest, you only see INFO and above logs