How to idiomatically open multiple managed resources from an object method in Python

Question

What is the most Pythonic way of constructing an object to open multiple (context managed) resources and do work with those resources?

I have a class which opens several managed resources, which are then operated on in class methods.

For example, if I had to open a connection to a local cache database and to a web server at the same time (e.g. check for data in the cache first, then pull from the server if not there).

I’ve been able to come up with some code to manage the resources using a yield statement, but it doesn’t seem very intuitive. In Python, is there a canonical way of approaching this problem?

Minimal example:

from contextlib import contextmanager

@contextmanager
def resource_A():
    print('opening resource A...')
    a = 'resource_A'
    yield a
    print('closing resource A...')

@contextmanager
def resource_B():
    print('opening resource B...')
    b = 'resource_B'
    yield b
    print('closing resource B...')

class ResourceManager(object):
    def opened(self):
        self.resources = self._resources()
        self.a, self.b = next(self.resources)
    def closed(self):
        try:
            next(self.resources)
        except StopIteration:
            del self.resources
            del self.a
            del self.b
    def _resources(self):
        with resource_A() as a, resource_B() as b:
            yield a, b
    def processing(self):
        print('doing something with resource_A and resource_B...')
    def __enter__(self):
        self.opened()
        return self
    def __exit__(self, ex_type, ex_val, traceback):
        self.closed()

Opening and closing

>>> r = ResourceManager()
>>> r.opened()
opening resource A...
opening resource B...
>>> r.a
'resource_A'
>>> r.b
'resource_B'
>>> r.closed()
closing resource B...
closing resource A...

Using with a context manager

>>> with ResourceManager() as r:
...     r.processing()
...
opening resource A...
opening resource B...
doing something with resource_A and resource_B...
closing resource B...
closing resource A...

The code above seems to work fine, but it doesn’t seem very intuitive. Specifically, the yield-next idiom seems a bit hard to digest.

Is there a better way to open/close multiple managed resources which will be subsequently used in class methods in Python?

Asked By: Chris Hubley

||

Source

Answer 1

I think, ExitStack will make your code easier
IMO, using __enter__ and __exit__ explicitly is more readable than next(...) thingy
Not actually about CMs, but big part of idiomatic python code includes naming. opened and closed reads as properties that should return boolean not be called, i.e. r.opened -> True. This is what most people would expect from your interface. Actions, on other hand, should be spelled as verbs, like open and close.

Simple example with ideas above:

class ResourceManager(object):
    def open(self):
        self.stack = ExitStack()
        self.a = self.stack.enter_context(resource_A())
        self.b = self.stack.enter_context(resource_B())

    def close(self):
        self.stack.close()

    def processing(self):
        print('doing something with resource_A and resource_B...')

    def __enter__(self):
        self.open()
        return self

    def __exit__(self, ex_type, ex_val, traceback):
        self.close()

Answered By: Slam

How to idiomatically open multiple managed resources from an object method in Python

Question:

Minimal example:

Opening and closing

Using with a context manager

Answers: