How do I skip a whole Python 'unittest' module at run time?
Question:
I would like my Python unittest module to tell the test runner to skip it entirety under some situations (such as being unable to import a module or locate a critical resource).
I can use @unittest.skipIf(...)
to skip a unittest.TestCase class, but how do I skip the entire module? Applying skips to every class is not sufficient because the class definitions themselves could cause exceptions if a module fails to import.
Answers:
It might be dirty to put all the unittest.TestCase
subclass definitions in a try...except
block, but it would work:
import unittest
try:
import eggs
class Spam(unittest.TestCase):
pass
class Ham(unittest.TestCase):
pass
# ...
except ImportError:
# print 'could not import eggs'
pass
None of the sub-classes would be defined if the eggs
import fails and all those classes (Spam, Ham, etc.
) get skipped. It would not be reflected in the output (good or bad depending on what you want).
If you look at the definition of unittest.skipIf
and unittest.skip
, you can see that the key is doing raise unittest.SkipTest(reason)
when the test is executed. If you’re okay with having it show up as one skipped test instead of several in the testrunner, you can simply raise unittest.SkipTest
yourself on import:
import unittest
try:
# do thing
except SomeException:
raise unittest.SkipTest("Such-and-such failed. Skipping all tests in foo.py")
Running with nosetests -v
gives:
Failure: SkipTest (Such-and-such failed. Skipping all tests in foo.py) ... SKIP:
Such-and-such failed. Skipping all tests in foo.py
----------------------------------------------------------------------
Ran 1 test in 0.002s
OK (SKIP=1)
Try defining a custom load_tests
function in your module:
import unittest
try:
(testcases)
except ImportError as e:
def load_tests(*args, **kwargs):
print("Failed to load tests: skipping")
return unittest.TestSuite() # no tests
After looking at the other answers here, this is the best answer I’ve come up with. It’s ugly, embedding the whole test suite in the exception handling, but it appears to do what you want. Specifically skipping the tests when the imports don’t work.
Assuming you’re talking about using nosetests -x for running the tests it should carry on past the tests that skip, at least it appeared to when I tried it.
import unittest
try:
import PyQt4
# the rest of the imports
# actual tests go here.
class TestDataEntryMixin(unittest.TestCase):
def test_somefeature(self):
# ....
except ImportError, e:
if e.message.find('PyQt4') >= 0:
class TestMissingDependency(unittest.TestCase):
@unittest.skip('Missing dependency - ' + e.message)
def test_fail():
pass
else:
raise
if __name__ == '__main__':
unittest.main()
If the import fails it replaces the test run with a single test that simply skips. I’ve also attempted to make sure that it doesn’t swallow any exceptions unintentionally. This solution owes a lot to all the other answers and comments to the question.
If you run it in verbose mode you will see this when it skips,
test_fail (test_openihm_gui_interface_mixins.TestMissingDependency) ... skipped 'Missing dependency - No module named PyQt4'
I found that using skipTest in setUp worked well. If you need a module imported, you use a try block to set e.g. module_failed = True, and in setUp call skipTest if it’s set. This reports the correct number of test skips with only a short try block needed:
import unittest
try:
import my_module
module_failed = False
except ImportError:
module_failed = True
class MyTests(unittest.TestCase):
def setUp(self):
if module_failed:
self.skipTest('module not tested')
def test_something(self):
#...
The solution proposed by otus works and is easier than the accepted solution in my opinion. But there is at least one downside. If you query my_module
in a decorator to skip a single test such as
@unittest.skipIf(my_module.support_foo, 'foo not supported')
def test_foo(self):
...
you will get a NameError: name 'my_module' is not defined
. The solution is put the skip inside the function definition:
def test_foo(self):
if not my_module.support_foo:
self.skipTest('foo not supported')
Combining the mentioned answers and using this answer:
import unittest
def module_exists(module_name):
try:
__import__(module_name)
except ImportError:
return False
else:
return True
class TestClass(unittest.TestCase):
@unittest.skipUnless(module_exists("moduleA"), 'ModuleA not installed')
def test_something(self):
# test something with moduleA
For Python 2.7+ (or using the unittest2 backport):
...
import unittest
def setUpModule():
try:
import something
except ImportError as err:
raise unittest.SkipTest(str(err))
class Tests(unittest.TestCase):
@classmethod
def setUpClass(cls):
try:
import something
except ImportError as err:
raise unittest.SkipTest(str(err))
...
@unittest.skip('comments_for_skipping_unit_tests')
class MyTests(unittest.TestCase):
def setUp(self):
pass
def test_something(self):
You can skip the whole unit test class by using the @unittest.skip decorator.
You can skip the whole class just like skipping individual methods. Documentation
@unittest.skip("showing class skipping")
class MySkippedTestCase(unittest.TestCase):
def test_not_run(self):
pass
So this question is nearly a decade old, and still applicable! However, none of the other answers addressed the problem I was having today – decorators on the tests coming from the missing library. In this case, it was hypothesis
and its decorators like @given
… So I was forced to whip up this less-than-friendly alternative:
try:
from hypothesis import assume, example, given, strategies as st
hypothesis_missing = False
except ImportError:
hypothesis_missing = True
def whatever(*args, **kwargs):
if args or kwargs:
return whatever
raise ImportError("Requires hypothesis library")
example = given = whatever
# Then this part is ickier to be py2.7 compatible
from six import add_move, MovedModule
add_move(MoveModule('mock', 'mock', 'unittest.mock'))
from six.moves import mock
st = mock.Mock() # Without this, things like st.integers() fail
A Python3-native solution would just tighten it up a little:
import unittest
st = unittest.mock.Mock()
Then on each testing class (ugh) I needed:
@unittest.skipIf(hypothesis_missing, "Requires hypothesis library")
IMPORTANT NOTE: Without the skipIf
, the tests will sometimes silently pass – the raise ImportError
in whatever()
only caught about half of them in my very limited testing.
If you are just looking to skip an entire module/submodule then place the following code in the relevant __init__.py
(e.g. tests/a/b/__init__.py
). Like the other solutions, this will have one message for all skipped tests.
import unittest
raise unittest.SkipTest("Don't run these tests because x, y, and z")
The SkipTest exception is the underlying method for telling Unittest to skip, and raising this in an __init__.py
causes that submodule (and all it’s subsubmodules) to be skipped.
This works with nose 1.3.7
I would like my Python unittest module to tell the test runner to skip it entirety under some situations (such as being unable to import a module or locate a critical resource).
I can use @unittest.skipIf(...)
to skip a unittest.TestCase class, but how do I skip the entire module? Applying skips to every class is not sufficient because the class definitions themselves could cause exceptions if a module fails to import.
It might be dirty to put all the unittest.TestCase
subclass definitions in a try...except
block, but it would work:
import unittest
try:
import eggs
class Spam(unittest.TestCase):
pass
class Ham(unittest.TestCase):
pass
# ...
except ImportError:
# print 'could not import eggs'
pass
None of the sub-classes would be defined if the eggs
import fails and all those classes (Spam, Ham, etc.
) get skipped. It would not be reflected in the output (good or bad depending on what you want).
If you look at the definition of unittest.skipIf
and unittest.skip
, you can see that the key is doing raise unittest.SkipTest(reason)
when the test is executed. If you’re okay with having it show up as one skipped test instead of several in the testrunner, you can simply raise unittest.SkipTest
yourself on import:
import unittest
try:
# do thing
except SomeException:
raise unittest.SkipTest("Such-and-such failed. Skipping all tests in foo.py")
Running with nosetests -v
gives:
Failure: SkipTest (Such-and-such failed. Skipping all tests in foo.py) ... SKIP:
Such-and-such failed. Skipping all tests in foo.py
----------------------------------------------------------------------
Ran 1 test in 0.002s
OK (SKIP=1)
Try defining a custom load_tests
function in your module:
import unittest
try:
(testcases)
except ImportError as e:
def load_tests(*args, **kwargs):
print("Failed to load tests: skipping")
return unittest.TestSuite() # no tests
After looking at the other answers here, this is the best answer I’ve come up with. It’s ugly, embedding the whole test suite in the exception handling, but it appears to do what you want. Specifically skipping the tests when the imports don’t work.
Assuming you’re talking about using nosetests -x for running the tests it should carry on past the tests that skip, at least it appeared to when I tried it.
import unittest
try:
import PyQt4
# the rest of the imports
# actual tests go here.
class TestDataEntryMixin(unittest.TestCase):
def test_somefeature(self):
# ....
except ImportError, e:
if e.message.find('PyQt4') >= 0:
class TestMissingDependency(unittest.TestCase):
@unittest.skip('Missing dependency - ' + e.message)
def test_fail():
pass
else:
raise
if __name__ == '__main__':
unittest.main()
If the import fails it replaces the test run with a single test that simply skips. I’ve also attempted to make sure that it doesn’t swallow any exceptions unintentionally. This solution owes a lot to all the other answers and comments to the question.
If you run it in verbose mode you will see this when it skips,
test_fail (test_openihm_gui_interface_mixins.TestMissingDependency) ... skipped 'Missing dependency - No module named PyQt4'
I found that using skipTest in setUp worked well. If you need a module imported, you use a try block to set e.g. module_failed = True, and in setUp call skipTest if it’s set. This reports the correct number of test skips with only a short try block needed:
import unittest
try:
import my_module
module_failed = False
except ImportError:
module_failed = True
class MyTests(unittest.TestCase):
def setUp(self):
if module_failed:
self.skipTest('module not tested')
def test_something(self):
#...
The solution proposed by otus works and is easier than the accepted solution in my opinion. But there is at least one downside. If you query my_module
in a decorator to skip a single test such as
@unittest.skipIf(my_module.support_foo, 'foo not supported')
def test_foo(self):
...
you will get a NameError: name 'my_module' is not defined
. The solution is put the skip inside the function definition:
def test_foo(self):
if not my_module.support_foo:
self.skipTest('foo not supported')
Combining the mentioned answers and using this answer:
import unittest
def module_exists(module_name):
try:
__import__(module_name)
except ImportError:
return False
else:
return True
class TestClass(unittest.TestCase):
@unittest.skipUnless(module_exists("moduleA"), 'ModuleA not installed')
def test_something(self):
# test something with moduleA
For Python 2.7+ (or using the unittest2 backport):
...
import unittest
def setUpModule():
try:
import something
except ImportError as err:
raise unittest.SkipTest(str(err))
class Tests(unittest.TestCase):
@classmethod
def setUpClass(cls):
try:
import something
except ImportError as err:
raise unittest.SkipTest(str(err))
...
@unittest.skip('comments_for_skipping_unit_tests')
class MyTests(unittest.TestCase):
def setUp(self):
pass
def test_something(self):
You can skip the whole unit test class by using the @unittest.skip decorator.
You can skip the whole class just like skipping individual methods. Documentation
@unittest.skip("showing class skipping")
class MySkippedTestCase(unittest.TestCase):
def test_not_run(self):
pass
So this question is nearly a decade old, and still applicable! However, none of the other answers addressed the problem I was having today – decorators on the tests coming from the missing library. In this case, it was hypothesis
and its decorators like @given
… So I was forced to whip up this less-than-friendly alternative:
try:
from hypothesis import assume, example, given, strategies as st
hypothesis_missing = False
except ImportError:
hypothesis_missing = True
def whatever(*args, **kwargs):
if args or kwargs:
return whatever
raise ImportError("Requires hypothesis library")
example = given = whatever
# Then this part is ickier to be py2.7 compatible
from six import add_move, MovedModule
add_move(MoveModule('mock', 'mock', 'unittest.mock'))
from six.moves import mock
st = mock.Mock() # Without this, things like st.integers() fail
A Python3-native solution would just tighten it up a little:
import unittest
st = unittest.mock.Mock()
Then on each testing class (ugh) I needed:
@unittest.skipIf(hypothesis_missing, "Requires hypothesis library")
IMPORTANT NOTE: Without the skipIf
, the tests will sometimes silently pass – the raise ImportError
in whatever()
only caught about half of them in my very limited testing.
If you are just looking to skip an entire module/submodule then place the following code in the relevant __init__.py
(e.g. tests/a/b/__init__.py
). Like the other solutions, this will have one message for all skipped tests.
import unittest
raise unittest.SkipTest("Don't run these tests because x, y, and z")
The SkipTest exception is the underlying method for telling Unittest to skip, and raising this in an __init__.py
causes that submodule (and all it’s subsubmodules) to be skipped.
This works with nose 1.3.7