Unittest teardown() del all attributes

Question

I have unit tests with setup and teardown methods which look like this:

def setUp(self):
   self.foo = "bar"
   self.bar = "foo"

def tearDown(self):
   del self.foo
   del self.bar

Is there a cleaner way to call __del__ on every object I instantiate in setUp? If I don’t do this then connections to a MySQL database which are opened in setUp (by instantiating those objects) seem to remain open after every test.

Ideally I would of course figure out the underlying problem (why those database connections aren’t being closed when the test finishes and the test case is discarded). In the meantime though, is there a cleaner way to del all those objects?

The database connections are created using the MySQLdb library, there is also a destructor to close the connection:

class Foo(object):

    def __init__(self,some_credentials):
        self.db_connection = MySQLdb.connect(some_credentials)

    def __del__(self):
        self.db_connection.close()

Asked By: Mike Vella

||

Source

Answer 1

You don’t have to delete all those attributes at all.

The TestCase instance will eventually be discarded after the tearDown; each test is run with a fresh, clean, shiny and above all empty new instance. Any attributes on the instance will be cleared, reference counts will drop once the test suite has run and if the instance was the only reference to those values, they’ll be gone from memory.

Quoting from the unittest.TestCase() documentation:

class unittest.TestCase(methodName='runTest')
Instances of the TestCase class represent the smallest testable units in the unittest universe. […]. Each instance of TestCase will run a single test method: the method named methodName.

Emphasis mine; a test runner will create these instances, passing in the name of the test method to run; if you have methods test_foo and test_bar, instances will be created passing in those names.

Do use the tearDown to clear up things outside the test instance; delete temporary files, remove mock patches, close database connections, etc. TestCase instances are only going to be finalised (removed from memory) once all tests have run, as the test runner may want to access each test later on to provide detail about them at the end of a full suite run.

Answered By: Martijn Pieters

Answer 2

The underlying problem here is that each python unit test does not discard the test instance after each test case is run. The instances are kept in memory so any object assigned to self is also kept in memory until the entire suite is complete.

You can reproduce this with the following code. The memory usage will grow with each additional test that is run. If self.large_list is set to None in the teardown then the memory usage remains consistent.

import resource
import unittest
import uuid


class TestSelfGarbageCollection(unittest.TestCase):

    def setUp(self):
        # Create an object that will use lots of memory
        self.large_list = []
        for _ in range(300000):
            self.large_list.append(uuid.uuid4())

    def tearDown(self):
        # Without the following line a copy of large_list will be kept in
        # memory for each test that runs, uncomment the line to allow the
        # large_list to be garbage collected.
        # self.large_list = None
        mb_memory = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1000
        print("Memory usage: %s MB" % mb_memory)

    def test_memory1(self):
        pass

    def test_memory2(self):
        pass

    def test_memory3(self):
        pass

    def test_memory4(self):
        pass

    def test_memory5(self):
        pass

Run with:

py.test test_large_memory.py -s -v

The simplest solution is to explicitly cleanup any large objects assigned to self or any objects that need cleanup (eg. database connections) in tearDown.

References:
Python’s leaky TestCase
Issue 11798: Test cases not garbage collected after run – Python tracker

Answered By: Kara

Answer 3

I always run tearDown(self) when I have setUp(self). This keeps prevents ‘kill 9’ errors from too much memory being accumulated. By setting variables to None, the total memory is cleared after every unittest.

class TestExample(unittest.TestCase):
    def setUp(self):
        self.example_var = "some small or large amount of data"

    def tearDown(self):
        self.example_var = None

Answered By: guocera

Unittest teardown() del all attributes

Question:

Answers: