Calling __enter__ and __exit__ manually
Question:
I’ve googled calling __enter__ manually
but with no luck. So let’s imagine I have MySQL connector class that uses __enter__
and __exit__
functions (originally used with with
statement) to connect/disconnect from a database.
And let’s have a class that uses 2 of these connections (for example for data sync). Note: this is not my real-life scenario, but it seems to be the simplest example.
Easiest way to make it all work together is class like this:
class DataSync(object):
def __init__(self):
self.master_connection = MySQLConnection(param_set_1)
self.slave_connection = MySQLConnection(param_set_2)
def __enter__(self):
self.master_connection.__enter__()
self.slave_connection.__enter__()
return self
def __exit__(self, exc_type, exc, traceback):
self.master_connection.__exit__(exc_type, exc, traceback)
self.slave_connection.__exit__(exc_type, exc, traceback)
# Some real operation functions
# Simple usage example
with DataSync() as sync:
records = sync.master_connection.fetch_records()
sync.slave_connection.push_records(records)
Q: Is it okay (is there anything wrong) to call __enter__
/__exit__
manually like this?
Pylint 1.1.0 didn’t issue any warnings on this, nor have I found any article about it (google link in the beggining).
And what about calling:
try:
# Db query
except MySQL.ServerDisconnectedException:
self.master_connection.__exit__(None, None, None)
self.master_connection.__enter__()
# Retry
Is this a good/bad practice? Why?
Answers:
Note: This answer doesn’t properly account for possible failures when there are multiple calls to underlying __enter__
and __exit__
methods. See Eric’s answer for one that does deal with that.
No, there’s nothing wrong with that. There are even places in the standard library that do it. Like the multiprocessing
module:
class SemLock(object):
def __init__(self, kind, value, maxvalue, *, ctx):
...
try:
sl = self._semlock = _multiprocessing.SemLock(
kind, value, maxvalue, self._make_name(),
unlink_now)
except FileExistsError:
pass
...
def __enter__(self):
return self._semlock.__enter__()
def __exit__(self, *args):
return self._semlock.__exit__(*args)
Or the tempfile
module:
class _TemporaryFileWrapper:
def __init__(self, file, name, delete=True):
self.file = file
self.name = name
self.delete = delete
self._closer = _TemporaryFileCloser(file, name, delete)
...
# The underlying __enter__ method returns the wrong object
# (self.file) so override it to return the wrapper
def __enter__(self):
self.file.__enter__()
return self
# Need to trap __exit__ as well to ensure the file gets
# deleted when used in a with statement
def __exit__(self, exc, value, tb):
result = self.file.__exit__(exc, value, tb)
self.close()
return result
The standard library examples aren’t calling __enter__
/__exit__
for two objects, but if you’ve got an object that’s responsible for creating/destroying the context for multiple objects instead of just one, calling __enter__
/__exit__
for all of them is fine.
The only potential gotcha is properly handling the return values of the __enter__
__exit__
calls for the objects you’re managing. With __enter__
, you need to make sure you’re returning whatever state
is required for the user of your wrapper object to get back from the with ... as <state>:
call. With __exit__
, you need to decide if you want to propagate any exception that occurred inside the context (by returning False
), or suppress it (by returning True
). Your managed objects could try to do it either way, you need to decide what makes sense for the wrapper object.
Your first example is not a good idea:
-
What happens if slave_connection.__enter__
throws an exception:
master_connection
acquires its resource
slave_connection
fails
DataSync.__enter__
propogates the exception
DataSync.__exit__
does not run
master_connection
is never cleaned up!
- Potential for Bad Things
-
What happens if master_connection.__exit__
throws an exception?
DataSync.__exit__
finished early
slave_connection
is never cleaned up!
- Potential for Bad Things
contextlib.ExitStack
can help here:
def __enter__(self):
with ExitStack() as stack:
stack.enter_context(self.master_connection)
stack.enter_context(self.slave_connection)
self._stack = stack.pop_all()
return self
def __exit__(self, exc_type, exc, traceback):
self._stack.__exit__(self, exc_type, exc, traceback)
Asking the same questions:
-
What happens if slave_connection.__enter__
throws an exception:
- The with block is exited, and
stack
cleans up master_connection
- Everything is ok!
-
What happens if master_connection.__exit__
throws an exception?
- Doesn’t matter,
slave_connection
gets cleaned up before this is called
- Everything is ok!
-
Ok, what happens if slave_connection.__exit__
throws an exception?
ExitStack
makes sure to call master_connection.__exit__
whatever happens to the slave connection
- Everything is ok!
There’s nothing wrong with calling __enter__
directly, but if you need to call it on more than one object, make sure you clean up properly!
I answer to the header only, i.e. to call __enter__
and __exit__
e.g. from an IPython prompt. You can do it this way:
ctx.__enter__()
ctx.__exit__(*sys.exc_info())
I’ve googled calling __enter__ manually
but with no luck. So let’s imagine I have MySQL connector class that uses __enter__
and __exit__
functions (originally used with with
statement) to connect/disconnect from a database.
And let’s have a class that uses 2 of these connections (for example for data sync). Note: this is not my real-life scenario, but it seems to be the simplest example.
Easiest way to make it all work together is class like this:
class DataSync(object):
def __init__(self):
self.master_connection = MySQLConnection(param_set_1)
self.slave_connection = MySQLConnection(param_set_2)
def __enter__(self):
self.master_connection.__enter__()
self.slave_connection.__enter__()
return self
def __exit__(self, exc_type, exc, traceback):
self.master_connection.__exit__(exc_type, exc, traceback)
self.slave_connection.__exit__(exc_type, exc, traceback)
# Some real operation functions
# Simple usage example
with DataSync() as sync:
records = sync.master_connection.fetch_records()
sync.slave_connection.push_records(records)
Q: Is it okay (is there anything wrong) to call __enter__
/__exit__
manually like this?
Pylint 1.1.0 didn’t issue any warnings on this, nor have I found any article about it (google link in the beggining).
And what about calling:
try:
# Db query
except MySQL.ServerDisconnectedException:
self.master_connection.__exit__(None, None, None)
self.master_connection.__enter__()
# Retry
Is this a good/bad practice? Why?
Note: This answer doesn’t properly account for possible failures when there are multiple calls to underlying __enter__
and __exit__
methods. See Eric’s answer for one that does deal with that.
No, there’s nothing wrong with that. There are even places in the standard library that do it. Like the multiprocessing
module:
class SemLock(object):
def __init__(self, kind, value, maxvalue, *, ctx):
...
try:
sl = self._semlock = _multiprocessing.SemLock(
kind, value, maxvalue, self._make_name(),
unlink_now)
except FileExistsError:
pass
...
def __enter__(self):
return self._semlock.__enter__()
def __exit__(self, *args):
return self._semlock.__exit__(*args)
Or the tempfile
module:
class _TemporaryFileWrapper:
def __init__(self, file, name, delete=True):
self.file = file
self.name = name
self.delete = delete
self._closer = _TemporaryFileCloser(file, name, delete)
...
# The underlying __enter__ method returns the wrong object
# (self.file) so override it to return the wrapper
def __enter__(self):
self.file.__enter__()
return self
# Need to trap __exit__ as well to ensure the file gets
# deleted when used in a with statement
def __exit__(self, exc, value, tb):
result = self.file.__exit__(exc, value, tb)
self.close()
return result
The standard library examples aren’t calling __enter__
/__exit__
for two objects, but if you’ve got an object that’s responsible for creating/destroying the context for multiple objects instead of just one, calling __enter__
/__exit__
for all of them is fine.
The only potential gotcha is properly handling the return values of the __enter__
__exit__
calls for the objects you’re managing. With __enter__
, you need to make sure you’re returning whatever state
is required for the user of your wrapper object to get back from the with ... as <state>:
call. With __exit__
, you need to decide if you want to propagate any exception that occurred inside the context (by returning False
), or suppress it (by returning True
). Your managed objects could try to do it either way, you need to decide what makes sense for the wrapper object.
Your first example is not a good idea:
-
What happens if
slave_connection.__enter__
throws an exception:master_connection
acquires its resourceslave_connection
failsDataSync.__enter__
propogates the exceptionDataSync.__exit__
does not runmaster_connection
is never cleaned up!- Potential for Bad Things
-
What happens if
master_connection.__exit__
throws an exception?DataSync.__exit__
finished earlyslave_connection
is never cleaned up!- Potential for Bad Things
contextlib.ExitStack
can help here:
def __enter__(self):
with ExitStack() as stack:
stack.enter_context(self.master_connection)
stack.enter_context(self.slave_connection)
self._stack = stack.pop_all()
return self
def __exit__(self, exc_type, exc, traceback):
self._stack.__exit__(self, exc_type, exc, traceback)
Asking the same questions:
-
What happens if
slave_connection.__enter__
throws an exception:- The with block is exited, and
stack
cleans upmaster_connection
- Everything is ok!
- The with block is exited, and
-
What happens if
master_connection.__exit__
throws an exception?- Doesn’t matter,
slave_connection
gets cleaned up before this is called - Everything is ok!
- Doesn’t matter,
-
Ok, what happens if
slave_connection.__exit__
throws an exception?ExitStack
makes sure to callmaster_connection.__exit__
whatever happens to the slave connection- Everything is ok!
There’s nothing wrong with calling __enter__
directly, but if you need to call it on more than one object, make sure you clean up properly!
I answer to the header only, i.e. to call __enter__
and __exit__
e.g. from an IPython prompt. You can do it this way:
ctx.__enter__()
ctx.__exit__(*sys.exc_info())