Python unittest AssertionError not being raised
Question:
I am trying to write unit tests for my Python package, and I am finding that, when I run the tests, AssertionErrors are not being raised, when they should be. Here is an MWE:
In exampleModule.py I have:
#! /usr/bin/env python
import unittest
class UnitTest(unittest.TestCase):
def runTest(self):
print("Starting test...")
a = 4
b = 5
self.assertEqual(a,b)
print("TEST COMPLETE")
return
And in testError.py I have:
#! /usr/bin/env python
import unittest
class AllTests(unittest.TestCase):
def testExample(self):
from exampleModule import UnitTest
UT = UnitTest()
UT.run()
return
if __name__ == '__main__':
unittest.main()
When I run testError.py I expect to see the AssertionError reported from the UnitTest in exampleModule.py, however, I simply see the following:
> ./testError.py
Starting test...
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
Why is the AssertionError not being raised? If I place the UnitTest() class in testError.py (i.e., have everything in the same file) then the AssertionError is raised. So why, when UnitTest is stored in a different file, does the error not get raised?
Answers:
The convention to structure unittests is like this:
Hold all tests in tests
directory.
test module will be named as test_modulea.py
with class TestClassA
nosetests -vw tests
to run all tests.
TestCase.run()
creates or updates a
TestResult
object, on the assumption you intend to do something interesting with those results.
But you immediately throw them away:
UT.run()
Any failures or errors—including exceptions raised—would have been in that results object.
For example:
def testExample(self):
from exampleModule import UnitTest
UT = UnitTest()
result = UT.run()
print('Errors: {!r}'.format(result.errors))
print('Failures: {!r}'.format(result.failures))
return
This prints:
Failures: [(<exampleModule.UnitTest testMethod=runTest>, ‘Traceback (most recent call last):n File "/home/kjc/tmp/exampleModule.py", line 11, in runTestn self.assertEqual(a, b)nAssertionError: 4 != 5n’)]
It catches all kinds of exceptions, like this example where I added sys.exit()
before the assertion:
Errors: [(<exampleModule.UnitTest testMethod=runTest>, ‘Traceback (most recent call last):n File "/home/kjc/tmp/exampleModule.py", line 11, in runTestn import sys; sys.exit()nSystemExitn’)]
For the record, the one passing test your example produced is testExample
itself.
You can verify this by calling self.fail()
early in testExample
.
(As one commentator said, calling unit tests from a unit test is a very strange thing to do.)
One Solution
You seem to want to run tests normally, but from multiple files.
If so, you can make a bunch of typical
unittest.main()
-style
test modules, and then load them manually.
Your test-everything script would look something like this:
#!/usr/bin/env python
import unittest
# Edit this to include new modules or TestCases to run.
_NAMES = ['testmoda', 'testmodb'] # testothermodule.SomeTestCase, etc...
def _main():
suite = unittest.defaultTestLoader.loadTestsFromNames(_NAMES)
unittest.TextTestRunner().run(suite)
return # WARNING: This always returns a successful exit value.
if __name__ == '__main__':
_main()
Your test modules would be the usual one-or-more TestCase
s, with conditional calls to unittest.main()
, so each could be run independently:
#!/usr/bin/env python
import unittest
class TestModA(unittest.TestCase):
def testThatPasses(self):
self.assertEqual(0, -0)
return
def testThatFails(self):
self.assertEqual(0, float('inf'))
return
def testThatBlowsUp(self):
raise RuntimeError('OMG!')
return
if '__main__' == __name__:
unittest.main()
If that’s not fiddly enough for you, you can also read up on the
discover()
method
or the module-level
load_tests
protocol.
I am trying to write unit tests for my Python package, and I am finding that, when I run the tests, AssertionErrors are not being raised, when they should be. Here is an MWE:
In exampleModule.py I have:
#! /usr/bin/env python
import unittest
class UnitTest(unittest.TestCase):
def runTest(self):
print("Starting test...")
a = 4
b = 5
self.assertEqual(a,b)
print("TEST COMPLETE")
return
And in testError.py I have:
#! /usr/bin/env python
import unittest
class AllTests(unittest.TestCase):
def testExample(self):
from exampleModule import UnitTest
UT = UnitTest()
UT.run()
return
if __name__ == '__main__':
unittest.main()
When I run testError.py I expect to see the AssertionError reported from the UnitTest in exampleModule.py, however, I simply see the following:
> ./testError.py
Starting test...
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
Why is the AssertionError not being raised? If I place the UnitTest() class in testError.py (i.e., have everything in the same file) then the AssertionError is raised. So why, when UnitTest is stored in a different file, does the error not get raised?
The convention to structure unittests is like this:
Hold all tests in tests
directory.
test module will be named as test_modulea.py
with class TestClassA
nosetests -vw tests
to run all tests.
TestCase.run()
creates or updates a
TestResult
object, on the assumption you intend to do something interesting with those results.
But you immediately throw them away:
UT.run()
Any failures or errors—including exceptions raised—would have been in that results object.
For example:
def testExample(self):
from exampleModule import UnitTest
UT = UnitTest()
result = UT.run()
print('Errors: {!r}'.format(result.errors))
print('Failures: {!r}'.format(result.failures))
return
This prints:
Failures: [(<exampleModule.UnitTest testMethod=runTest>, ‘Traceback (most recent call last):n File "/home/kjc/tmp/exampleModule.py", line 11, in runTestn self.assertEqual(a, b)nAssertionError: 4 != 5n’)]
It catches all kinds of exceptions, like this example where I added sys.exit()
before the assertion:
Errors: [(<exampleModule.UnitTest testMethod=runTest>, ‘Traceback (most recent call last):n File "/home/kjc/tmp/exampleModule.py", line 11, in runTestn import sys; sys.exit()nSystemExitn’)]
For the record, the one passing test your example produced is testExample
itself.
You can verify this by calling self.fail()
early in testExample
.
(As one commentator said, calling unit tests from a unit test is a very strange thing to do.)
One Solution
You seem to want to run tests normally, but from multiple files.
If so, you can make a bunch of typical
unittest.main()
-style
test modules, and then load them manually.
Your test-everything script would look something like this:
#!/usr/bin/env python
import unittest
# Edit this to include new modules or TestCases to run.
_NAMES = ['testmoda', 'testmodb'] # testothermodule.SomeTestCase, etc...
def _main():
suite = unittest.defaultTestLoader.loadTestsFromNames(_NAMES)
unittest.TextTestRunner().run(suite)
return # WARNING: This always returns a successful exit value.
if __name__ == '__main__':
_main()
Your test modules would be the usual one-or-more TestCase
s, with conditional calls to unittest.main()
, so each could be run independently:
#!/usr/bin/env python
import unittest
class TestModA(unittest.TestCase):
def testThatPasses(self):
self.assertEqual(0, -0)
return
def testThatFails(self):
self.assertEqual(0, float('inf'))
return
def testThatBlowsUp(self):
raise RuntimeError('OMG!')
return
if '__main__' == __name__:
unittest.main()
If that’s not fiddly enough for you, you can also read up on the
discover()
method
or the module-level
load_tests
protocol.