how to add a package to sys path for testing
Question:
This question is occasioned by instructions in the python guide for adding a project to sys path to use in tests, which do not seem to work unless I am misunderstanding the instructions
I have a directory structure for a python project like this
sample/a.py
sample/b.py
sample/c.py
sample/__init__.py
test/context.py
test/test_something.py
test/__init__.py
docs
According to the python guide, I should create a test/context.py file and add this
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
import sample
Then, in my test/test_something.py file, it says I can do this
from .context import sample
The guide says “This will always work as expected”.
but, when I cd into test and run
python -m unittest test_something
I get an error
ValueError: Attempted relative import in non-package
and the error message specifically refers to this: from .context import sample
Question: How can I add my sample package to the sys path correctly?
When answering, can you also clarify if the solution will handle absolute imports within the sample package. For example, my sample.a imports sample.b etc. When I had my tests structured a different way, I did an absolute import of sample.a, but since it has a relative import of from .b import Boo, it produced a similar error
Update
`File "/usr/local/lib/python2.7/runpy.py", line 162 in _run_module_as_main "__main__", fname, loader, pkg_name)
File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals
File "/usr/local/lib/python2.7/unittest/__main__.py", line 12, in module main(module=None)
File "/usr/local/lib/python2.7/unittest/main.py", line 94, in __init__ self.parseArgs(argv)
File "/usr/local/lib/python2.7/unittest/main.py", line 149 in parseArgs self.createTests()
File "/usr/local/lib/python2.7/unittest/main.py", line 158, in createTests self.module)
File "/usr/local/lib/python2.7/unittest/loader.py", line 130, in loadTestsFromNames suites = [self.loadTestsFromName(name,module) for name in names]
File "/usr/local/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName module = __import__('-'.join(parts_copy))
File "test_something.py", line 8, in module from .context import sample
Update
if I run the following command from root directory
python -m unittest test
It says , “Ran 0 tests in 0.000s”
If, as was suggested in the comments by @cuongnv, I run this from root directory
python -m unittest test/test_something.py
or this (without the file extension)
python -m unittest test/test_something
It says “Import by filename is not supported”
Answers:
Try adding an empty __init__.py to tests/: touch tests/__init__.py should do it.
Question: How can I add my sample package to the sys path correctly?
You’re doing it the right way, but you missed declaring your folder to be a package. Try solution of Christian, it should work.
Your path is stored in sys.path. By doing this:
sys.path.insert(0, os.path.abspath('..'))
You’re telling your python to add upper folder (of current file) into your path. As sys.path is a list, you can using other methods of list like insert, append…
In your case, you’re inserting your upper dir at top of the path list.
See:
In [1]: import sys
In [2]: sys.path
Out[2]:
['',
'/usr/local/bin',
'/usr/lib/python3.4',
'/usr/lib/python3.4/plat-x86_64-linux-gnu',
'/usr/lib/python3.4/lib-dynload',
'/usr/local/lib/python3.4/dist-packages',
'/usr/lib/python3/dist-packages',
'/usr/local/lib/python3.4/dist-packages/IPython/extensions',
'/home/cuong/.ipython']
In [3]: sys.path.insert(0, '/tmp/foo')
In [4]: sys.path
Out[4]:
['/tmp/foo', **<-- on top**
'',
'/usr/local/bin',
'/usr/lib/python3.4',
'/usr/lib/python3.4/plat-x86_64-linux-gnu',
'/usr/lib/python3.4/lib-dynload',
'/usr/local/lib/python3.4/dist-packages',
'/usr/lib/python3/dist-packages',
'/usr/local/lib/python3.4/dist-packages/IPython/extensions',
'/home/cuong/.ipython']
So, from here, when you have
import sample
your python will try to look in path to see if there is any sample package.
Unfortunately, it can’t find sample as you didn’t make it as a package because your forgot __init__.py in sample folder.
Hope my explanation would help you to understand and you can handle other situations different to this.
I had a battle to get my testing directory structure to work outside of an
IDE. Please find my solution below. Tested on
Windows 7 using python 3.6 and Linux Mint using python 3.4, running the code
using the command line:
python -m pytest test_compress_files.py
The file I wrote to be tested is called compress_files.py in a directory named
src. The file containing tests to be run using pytest is called
test_compress_files.py in a subdirectory tests, so the full directory path is
srctests. I needed to add a file called context.py to the srctests
directory. This file is used in test_compress_files.py to enable access to compress_files.py in the directory above. The _init_.py files are empty.
Directory structure
src
__init__.py
compress_files.py
srctests
__init__.py
context.py
test_compress_files.py
compress_files.py contains the script to be tested.
context.py:
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
import compress_files
The line:
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
comes from the suggestion at the hitch hikers guide to python at http://docs.python-guide.org/en/latest/writing/structure/. This adds the path of the directory above the /src/tests directory to sys.path, which in this case is /src.
test_compress_files.py:
import os
import pytest
from context import compress_files
from compress_files import *
# tests start here
...
I can’t comment yet but i was trying out Oppy’s answers in python 3.7 env and it fail as:
ImportError: cannot import name 'compress_files' from '__main__' (test_compress_files.py)
I resolved using a modified test_compress_files.py (notice the removed dot) :
import os
import pytest
from context import compress_files
from compress_files import *
# tests start here
...
This is a bit old, and slightly tangential, but this question/answers helped me when trying to patch on pandas.
I had imported pandas the "standard" way as pd:
import pandas as pd
So when declaring the import path I had to do:
unittest.mock.patch("src.path.to.file.pd")
NOT
unittest.mock.patch("src.path.to.file.pandas")
This question is occasioned by instructions in the python guide for adding a project to sys path to use in tests, which do not seem to work unless I am misunderstanding the instructions
I have a directory structure for a python project like this
sample/a.py
sample/b.py
sample/c.py
sample/__init__.py
test/context.py
test/test_something.py
test/__init__.py
docs
According to the python guide, I should create a test/context.py file and add this
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
import sample
Then, in my test/test_something.py file, it says I can do this
from .context import sample
The guide says “This will always work as expected”.
but, when I cd into test and run
python -m unittest test_something
I get an error
ValueError: Attempted relative import in non-package
and the error message specifically refers to this: from .context import sample
Question: How can I add my sample package to the sys path correctly?
When answering, can you also clarify if the solution will handle absolute imports within the sample package. For example, my sample.a imports sample.b etc. When I had my tests structured a different way, I did an absolute import of sample.a, but since it has a relative import of from .b import Boo, it produced a similar error
Update
`File "/usr/local/lib/python2.7/runpy.py", line 162 in _run_module_as_main "__main__", fname, loader, pkg_name)
File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals
File "/usr/local/lib/python2.7/unittest/__main__.py", line 12, in module main(module=None)
File "/usr/local/lib/python2.7/unittest/main.py", line 94, in __init__ self.parseArgs(argv)
File "/usr/local/lib/python2.7/unittest/main.py", line 149 in parseArgs self.createTests()
File "/usr/local/lib/python2.7/unittest/main.py", line 158, in createTests self.module)
File "/usr/local/lib/python2.7/unittest/loader.py", line 130, in loadTestsFromNames suites = [self.loadTestsFromName(name,module) for name in names]
File "/usr/local/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName module = __import__('-'.join(parts_copy))
File "test_something.py", line 8, in module from .context import sample
Update
if I run the following command from root directory
python -m unittest test
It says , “Ran 0 tests in 0.000s”
If, as was suggested in the comments by @cuongnv, I run this from root directory
python -m unittest test/test_something.py
or this (without the file extension)
python -m unittest test/test_something
It says “Import by filename is not supported”
Try adding an empty __init__.py to tests/: touch tests/__init__.py should do it.
Question: How can I add my sample package to the sys path correctly?
You’re doing it the right way, but you missed declaring your folder to be a package. Try solution of Christian, it should work.
Your path is stored in sys.path. By doing this:
sys.path.insert(0, os.path.abspath('..'))
You’re telling your python to add upper folder (of current file) into your path. As sys.path is a list, you can using other methods of list like insert, append…
In your case, you’re inserting your upper dir at top of the path list.
See:
In [1]: import sys
In [2]: sys.path
Out[2]:
['',
'/usr/local/bin',
'/usr/lib/python3.4',
'/usr/lib/python3.4/plat-x86_64-linux-gnu',
'/usr/lib/python3.4/lib-dynload',
'/usr/local/lib/python3.4/dist-packages',
'/usr/lib/python3/dist-packages',
'/usr/local/lib/python3.4/dist-packages/IPython/extensions',
'/home/cuong/.ipython']
In [3]: sys.path.insert(0, '/tmp/foo')
In [4]: sys.path
Out[4]:
['/tmp/foo', **<-- on top**
'',
'/usr/local/bin',
'/usr/lib/python3.4',
'/usr/lib/python3.4/plat-x86_64-linux-gnu',
'/usr/lib/python3.4/lib-dynload',
'/usr/local/lib/python3.4/dist-packages',
'/usr/lib/python3/dist-packages',
'/usr/local/lib/python3.4/dist-packages/IPython/extensions',
'/home/cuong/.ipython']
So, from here, when you have
import sample
your python will try to look in path to see if there is any sample package.
Unfortunately, it can’t find sample as you didn’t make it as a package because your forgot __init__.py in sample folder.
Hope my explanation would help you to understand and you can handle other situations different to this.
I had a battle to get my testing directory structure to work outside of an
IDE. Please find my solution below. Tested on
Windows 7 using python 3.6 and Linux Mint using python 3.4, running the code
using the command line:
python -m pytest test_compress_files.py
The file I wrote to be tested is called compress_files.py in a directory named
src. The file containing tests to be run using pytest is called
test_compress_files.py in a subdirectory tests, so the full directory path is
srctests. I needed to add a file called context.py to the srctests
directory. This file is used in test_compress_files.py to enable access to compress_files.py in the directory above. The _init_.py files are empty.
Directory structure
src
__init__.py
compress_files.py
srctests
__init__.py
context.py
test_compress_files.py
compress_files.py contains the script to be tested.
context.py:
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
import compress_files
The line:
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
comes from the suggestion at the hitch hikers guide to python at http://docs.python-guide.org/en/latest/writing/structure/. This adds the path of the directory above the /src/tests directory to sys.path, which in this case is /src.
test_compress_files.py:
import os
import pytest
from context import compress_files
from compress_files import *
# tests start here
...
I can’t comment yet but i was trying out Oppy’s answers in python 3.7 env and it fail as:
ImportError: cannot import name 'compress_files' from '__main__' (test_compress_files.py)
I resolved using a modified test_compress_files.py (notice the removed dot) :
import os
import pytest
from context import compress_files
from compress_files import *
# tests start here
...
This is a bit old, and slightly tangential, but this question/answers helped me when trying to patch on pandas.
I had imported pandas the "standard" way as pd:
import pandas as pd
So when declaring the import path I had to do:
unittest.mock.patch("src.path.to.file.pd")
NOT
unittest.mock.patch("src.path.to.file.pandas")