changing order of unit tests in Python

Question:

How can I make it so unit tests in Python (using unittest) are run in the order in which they are specified in the file?

Asked By: user248237

||

Answers:

Clever Naming.

class Test01_Run_Me_First( unittest.TestCase ):
    def test010_do_this( self ):
        assertTrue( True )
    def test020_do_that( self ):
        etc.

Is one way to force a specific order.

Answered By: S.Lott

There are also test runners which do that by themselves – I think py.test does it.

Answered By: Felix Schwarz

You can change the default sorting behavior by setting a custom comparison function. In unittest.py you can find the class variable unittest.TestLoader.sortTestMethodsUsing which is set to the builtin function cmp by default.

For example you can revert the execution order of your tests with doing this:

import unittest
unittest.TestLoader.sortTestMethodsUsing = lambda _, x, y: cmp(y, x)
Answered By: atomocopter

Use proboscis library as I mentioned already (please see short description there).

Answered By: Sergei Danielian

As said above, normally tests in test cases should be tested in any (i.e. random) order.

However, if you do want to order the tests in the test case, apparently it is not trivial.
Tests (method names) are retrieved from test cases using dir(MyTest), which returns a sorted list of members. You can use a clever (?) hack to order methods by their line numbers. This will work for one test case:

if __name__ == "__main__":
    loader = unittest.TestLoader()
    ln = lambda f: getattr(MyTestCase, f).im_func.func_code.co_firstlineno
    lncmp = lambda a, b: cmp(ln(a), ln(b))
    loader.sortTestMethodsUsing = lncmp
    unittest.main(testLoader=loader, verbosity=2)
Answered By: Udi

I found a solution for it using PyTest ordering plugin provided here.

Try py.test YourModuleName.py -vv in CLI and the test will run in the order they have appeared in your module.

I did the same thing and works fine for me.

Note: You need to install PyTest package and import it.

Answered By: Mahsa Mortazavi

hacky way (run this file in pycharm or other unit test runner)

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import unittest


def make_suite():
    class Test(unittest.TestCase):
        def test_32(self):
            print "32"
        def test_23(self):
            print "23"

    suite = unittest.TestSuite()
    suite.addTest(Test('test_32'))
    suite.addTest(Test('test_23'))
    return suite

suite = make_suite()

class T(unittest.TestCase):
    counter = 0
    def __call__(self, *args, **kwargs):
        res = suite._tests[T.counter](*args, **kwargs)
        T.counter += 1
        return res

for t in suite._tests:
    name = "{}${}".format(t._testMethodName, t.__class__.__name__)
    setattr(T, name, t)
Answered By: user8214838

The default order is alphabetical. You can put test_<int> before every test to specify the order of execution.

Also you can set unittest.TestLoader.sortTestMethodsUsing = None to eliminate the sort.

Check out Unittest Documentation for more details.

Answered By: Ognian Baruh
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.