preventing python coverage from including virtual environment site packages
Question:
I am new to coverage and ran into a strange problem. My coverage is taking my virtual environment site packages into account.
Here is the output of the coverage run:
coverage run test.py
....................
----------------------------------------------------------------------
Ran 20 tests in 0.060s
OK
(atcatalog)- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -jmfrank63@fullstack-audio-text-catalog:~/workspace (git master) [19:58:45]
$ coverage report
Name Stmts Miss Cover
-----------------------------------------------------------------------------------------------------
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/__init__ 18 0 100%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/_compat 38 20 47%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/app 528 255 52%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/blueprints 156 118 24%
.
.
.
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/urls 412 215 48%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/utils 242 175 28%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/wrappers 568 298 48%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/wsgi 448 352 21%
atcatalog/__init__ 7 0 100%
atcatalog/views/__init__ 0 0 100%
atcatalog/views/publang 7 0 100%
atcatalog/views/pubtext 1 0 100%
atcatalog/views/userlang 13 0 100%
atcatalog/views/users 5 0 100%
atcatalog/views/usertext 14 0 100%
test 120 0 100%
-----------------------------------------------------------------------------------------------------
TOTAL 12530 8044 36%
(atcatalog)- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -jmfrank63@fullstack-audio-text-catalog:~/workspace (git master) [19:58:55]
Here is the structure of my project directory which resides under home:
workspace/
├── README.md
├── atcatalog
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── static
│ ├── templates
│ └── views
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── publang.py
│ ├── publang.pyc
│ ├── pubtext.py
│ ├── pubtext.pyc
│ ├── userlang.py
│ ├── userlang.pyc
│ ├── users.py
│ ├── users.pyc
│ ├── usertext.py
│ └── usertext.pyc
├── requirements.txt
├── run.py
└── test.py
I had the virtual environment at first inside the project directory and now moved it out to ~/Envs with virtualenvwrapper, but the problem persisted.
run.py and test.py are not special in any way, they both import app from atcatalog.
I also tried to find ways to omit the virtual environment directory, but google gave no answer (surprisingly).
I don’t think it is the purpose of coverage to test already well tested site-packages. So I would exclude them from the run.
How can I accomplish to avoid coverage having testing my site-packages?
Answers:
Try using py.test and then specifiying your test options in a setup.cfg file. You will need to pip install pytest first.
For example:
[pytest]
norecursedirs = build docs/_build *.egg .tox *.venv
python_files = tests/functional* tests/integration*
addopts =
#--verbose
--tb short
# Turn on --capture to have brief, less noisy output
# You will only see output if the test fails
# Use --capture no if you want to see it all or have problems debugging
--capture fd
# --capture no
# show extra test summary info as specified by chars (f)ailed, (E)error, (s)skipped, (x)failed, (X)passed.
- rfEsxX
--junitxml junit.xml
--cov workspace --cov-report xml --cov-report term-missing
You can read more about configuring py.test here: https://pytest.org/latest/customize.html
Thanks to tknickman I figured it out: Use either
coverage run --source <path to project dir> test.py
or create a configuration file .coveragerc which resides in the directory you run coverage from, with the following content:
[run]
source =
<path to project dir>
This provides you do not have your virtual environment installed under the project directory.
If you have the virtual environment installed under the project dir you can use
coverage run --source <project path> --omit <pattern> test.py
Note that omit wants a file pattern like
~/projectdir/venv/*
instead of a path.
The corresponding .coveragerc would look like this:
[run]
source=
<path to project dir>
omit=
<path to project dir>/<name of virtual env>/*
I still think that like packages of the standard library any packages installed under site-packages should not be covered by default.
If using pytest, you can specify exclusive paths or files to test in setup.cfg (see docs):
[pytest]
# a directory
testpaths = tests
# exact file(s)
python_files = tests/test1.py tests/test2.py
It looks like if you include the python_files and testpaths parameters, then the python_files will only be used.
In your setup.cfg file include:
[coverage:run]
omit=*/site-packages/*,*/tests/*,*/.eggs/*
Or any other folders that show up in your results that you want to hide from coverage.
I am new to coverage and ran into a strange problem. My coverage is taking my virtual environment site packages into account.
Here is the output of the coverage run:
coverage run test.py
....................
----------------------------------------------------------------------
Ran 20 tests in 0.060s
OK
(atcatalog)- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -jmfrank63@fullstack-audio-text-catalog:~/workspace (git master) [19:58:45]
$ coverage report
Name Stmts Miss Cover
-----------------------------------------------------------------------------------------------------
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/__init__ 18 0 100%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/_compat 38 20 47%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/app 528 255 52%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/blueprints 156 118 24%
.
.
.
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/urls 412 215 48%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/utils 242 175 28%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/wrappers 568 298 48%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/wsgi 448 352 21%
atcatalog/__init__ 7 0 100%
atcatalog/views/__init__ 0 0 100%
atcatalog/views/publang 7 0 100%
atcatalog/views/pubtext 1 0 100%
atcatalog/views/userlang 13 0 100%
atcatalog/views/users 5 0 100%
atcatalog/views/usertext 14 0 100%
test 120 0 100%
-----------------------------------------------------------------------------------------------------
TOTAL 12530 8044 36%
(atcatalog)- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -jmfrank63@fullstack-audio-text-catalog:~/workspace (git master) [19:58:55]
Here is the structure of my project directory which resides under home:
workspace/
├── README.md
├── atcatalog
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── static
│ ├── templates
│ └── views
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── publang.py
│ ├── publang.pyc
│ ├── pubtext.py
│ ├── pubtext.pyc
│ ├── userlang.py
│ ├── userlang.pyc
│ ├── users.py
│ ├── users.pyc
│ ├── usertext.py
│ └── usertext.pyc
├── requirements.txt
├── run.py
└── test.py
I had the virtual environment at first inside the project directory and now moved it out to ~/Envs with virtualenvwrapper, but the problem persisted.
run.py and test.py are not special in any way, they both import app from atcatalog.
I also tried to find ways to omit the virtual environment directory, but google gave no answer (surprisingly).
I don’t think it is the purpose of coverage to test already well tested site-packages. So I would exclude them from the run.
How can I accomplish to avoid coverage having testing my site-packages?
Try using py.test and then specifiying your test options in a setup.cfg file. You will need to pip install pytest first.
For example:
[pytest]
norecursedirs = build docs/_build *.egg .tox *.venv
python_files = tests/functional* tests/integration*
addopts =
#--verbose
--tb short
# Turn on --capture to have brief, less noisy output
# You will only see output if the test fails
# Use --capture no if you want to see it all or have problems debugging
--capture fd
# --capture no
# show extra test summary info as specified by chars (f)ailed, (E)error, (s)skipped, (x)failed, (X)passed.
- rfEsxX
--junitxml junit.xml
--cov workspace --cov-report xml --cov-report term-missing
You can read more about configuring py.test here: https://pytest.org/latest/customize.html
Thanks to tknickman I figured it out: Use either
coverage run --source <path to project dir> test.py
or create a configuration file .coveragerc which resides in the directory you run coverage from, with the following content:
[run]
source =
<path to project dir>
This provides you do not have your virtual environment installed under the project directory.
If you have the virtual environment installed under the project dir you can use
coverage run --source <project path> --omit <pattern> test.py
Note that omit wants a file pattern like
~/projectdir/venv/*
instead of a path.
The corresponding .coveragerc would look like this:
[run]
source=
<path to project dir>
omit=
<path to project dir>/<name of virtual env>/*
I still think that like packages of the standard library any packages installed under site-packages should not be covered by default.
If using pytest, you can specify exclusive paths or files to test in setup.cfg (see docs):
[pytest]
# a directory
testpaths = tests
# exact file(s)
python_files = tests/test1.py tests/test2.py
It looks like if you include the python_files and testpaths parameters, then the python_files will only be used.
In your setup.cfg file include:
[coverage:run]
omit=*/site-packages/*,*/tests/*,*/.eggs/*
Or any other folders that show up in your results that you want to hide from coverage.