How to handle both `with open(…)` and `sys.stdout` nicely?
Question:
Often I need to output data either to file or, if file is not specified, to stdout. I use the following snippet:
if target:
with open(target, 'w') as h:
h.write(content)
else:
sys.stdout.write(content)
I would like to rewrite it and handle both targets uniformly.
In ideal case it would be:
with open(target, 'w') as h:
h.write(content)
but this will not work well because sys.stdout is be closed when leaving with
block and I don’t want that. I neither want to
stdout = open(target, 'w')
...
because I would need to remember to restore original stdout.
Related:
- Redirect stdout to a file in Python?
- Handling Exceptions – interesting article about handling exceptions in Python, as compared to C++
Edit
I know that I can wrap target
, define separate function or use context manager. I look for a simple, elegant, idiomatic solution fitting that wouldn’t require more than 5 lines
Answers:
Why LBYL when you can EAFP?
try:
with open(target, 'w') as h:
h.write(content)
except TypeError:
sys.stdout.write(content)
Why rewrite it to use the with
/as
block uniformly when you have to make it work in a convoluted way? You’ll add more lines and reduce performance.
Just thinking outside of the box here, how about a custom open()
method?
import sys
import contextlib
@contextlib.contextmanager
def smart_open(filename=None):
if filename and filename != '-':
fh = open(filename, 'w')
else:
fh = sys.stdout
try:
yield fh
finally:
if fh is not sys.stdout:
fh.close()
Use it like this:
# For Python 2 you need this line
from __future__ import print_function
# writes to some_file
with smart_open('some_file') as fh:
print('some output', file=fh)
# writes to stdout
with smart_open() as fh:
print('some output', file=fh)
# writes to stdout
with smart_open('-') as fh:
print('some output', file=fh)
Stick with your current code. It’s simple and you can tell exactly what it’s doing just by glancing at it.
Another way would be with an inline if
:
handle = open(target, 'w') if target else sys.stdout
handle.write(content)
if handle is not sys.stdout:
handle.close()
But that isn’t much shorter than what you have and it looks arguably worse.
You could also make sys.stdout
unclosable, but that doesn’t seem too Pythonic:
sys.stdout.close = lambda: None
with (open(target, 'w') if target else sys.stdout) as handle:
handle.write(content)
I’d also go for a simple wrapper function, which can be pretty simple if you can ignore the mode (and consequently stdin vs. stdout), for example:
from contextlib import contextmanager
import sys
@contextmanager
def open_or_stdout(filename):
if filename != '-':
with open(filename, 'w') as f:
yield f
else:
yield sys.stdout
If you really must insist on something more “elegant”, i.e. a one-liner:
>>> import sys
>>> target = "foo.txt"
>>> content = "foo"
>>> (lambda target, content: (lambda target, content: filter(lambda h: not h.write(content), (target,))[0].close())(open(target, 'w'), content) if target else sys.stdout.write(content))(target, content)
foo.txt
appears and contains the text foo
.
Okay, if we are getting into one-liner wars, here’s:
(target and open(target, 'w') or sys.stdout).write(content)
I like Jacob’s original example as long as context is only written in one place. It would be a problem if you end up re-opening the file for many writes. I think I would just make the decision once at the top of the script and let the system close the file on exit:
output = target and open(target, 'w') or sys.stdout
...
output.write('thing onen')
...
output.write('thing twon')
You could include your own exit handler if you think its more tidy
import atexit
def cleanup_output():
global output
if output is not sys.stdout:
output.close()
atexit(cleanup_output)
Another possible solution: do not try to avoid the context manager exit method, just duplicate stdout.
with (os.fdopen(os.dup(sys.stdout.fileno()), 'w')
if target == '-'
else open(target, 'w')) as f:
f.write("Foo")
How about opening a new fd for sys.stdout? This way you won’t have any problems closing it:
if not target:
target = "/dev/stdout"
with open(target, 'w') as f:
f.write(content)
if (out != sys.stdout):
with open(out, 'wb') as f:
f.write(data)
else:
out.write(data)
Slight improvement in some cases.
An improvement of Wolph’s answer
import sys
import contextlib
@contextlib.contextmanager
def smart_open(filename: str, mode: str = 'r', *args, **kwargs):
'''Open files and i/o streams transparently.'''
if filename == '-':
if 'r' in mode:
stream = sys.stdin
else:
stream = sys.stdout
if 'b' in mode:
fh = stream.buffer # type: IO
else:
fh = stream
close = False
else:
fh = open(filename, mode, *args, **kwargs)
close = True
try:
yield fh
finally:
if close:
try:
fh.close()
except AttributeError:
pass
This allows binary IO and pass eventual extraneous arguments to open
if filename
is indeed a file name.
import contextlib
import sys
with contextlib.ExitStack() as stack:
h = stack.enter_context(open(target, 'w')) if target else sys.stdout
h.write(content)
Just two extra lines if you’re using Python 3.3 or higher: one line for the extra import
and one line for the stack.enter_context
.
If it’s fine that sys.stdout
is closed after with
body, you can also use patterns like this:
# Use stdout when target is "-"
with open(target, "w") if target != "-" else sys.stdout as f:
f.write("hello world")
# Use stdout when target is falsy (None, empty string, ...)
with open(target, "w") if target else sys.stdout as f:
f.write("hello world")
or even more generally:
with target if isinstance(target, io.IOBase) else open(target, "w") as f:
f.write("hello world")
The following solution is not a beauty, but from a time long, long ago; just before with …
handler = open(path, mode = 'a') if path else sys.stdout
try:
print('stuff', file = handler)
... # other stuff or more writes/prints, etc.
except Exception as e:
if not (path is None): handler.close()
raise e
handler.close()
One way to solve it is with polymorphism. Pathlib.path
has an open
method that functions as you would expect:
from pathlib import Path
output = Path("/path/to/file.csv")
with output.open(mode="w", encoding="utf-8") as f:
print("hello world", file=f)
we can copy this interface for printing
import sys
class Stdout:
def __init__(self, *args):
pass
def open(self, mode=None, encoding=None):
return self
def __enter__(self):
return sys.stdout
def __exit__(self, exc_type, exc_value, traceback):
pass
Now we simply replace Path
with Stdout
output = Stdout("/path/to/file.csv")
with output.open(mode="w", encoding="utf-8") as f:
print("hello world", file=f)
This isn’t necessarily better than overloading open
, but it’s a convenient solution if you’re using Path
objects.
With python 3 you can used wrap stdout
file descriptor with IO object and avoid closing on context leave it with closefd=False
:
h = open(target, 'w') if target else open(sys.stdout.fileno(), 'w', closefd=False)
with h as h:
h.write(content)
As pointed in Conditional with statement in Python, Python 3.7 allows using contextlib.nullcontext for that:
from contextlib import nullcontext
with open(target, "w") if target else nullcontext(sys.stdout) as f:
f.write(content)
This is a simpler and shorter version of the accepted answer
import contextlib, sys
def writer(fn):
@contextlib.contextmanager
def stdout():
yield sys.stdout
return open(fn, 'w') if fn else stdout()
usage:
with writer('') as w:
w.write('hellon')
with writer('file.txt') as w:
w.write('hellon')
Often I need to output data either to file or, if file is not specified, to stdout. I use the following snippet:
if target:
with open(target, 'w') as h:
h.write(content)
else:
sys.stdout.write(content)
I would like to rewrite it and handle both targets uniformly.
In ideal case it would be:
with open(target, 'w') as h:
h.write(content)
but this will not work well because sys.stdout is be closed when leaving with
block and I don’t want that. I neither want to
stdout = open(target, 'w')
...
because I would need to remember to restore original stdout.
Related:
- Redirect stdout to a file in Python?
- Handling Exceptions – interesting article about handling exceptions in Python, as compared to C++
Edit
I know that I can wrap target
, define separate function or use context manager. I look for a simple, elegant, idiomatic solution fitting that wouldn’t require more than 5 lines
Why LBYL when you can EAFP?
try:
with open(target, 'w') as h:
h.write(content)
except TypeError:
sys.stdout.write(content)
Why rewrite it to use the with
/as
block uniformly when you have to make it work in a convoluted way? You’ll add more lines and reduce performance.
Just thinking outside of the box here, how about a custom open()
method?
import sys
import contextlib
@contextlib.contextmanager
def smart_open(filename=None):
if filename and filename != '-':
fh = open(filename, 'w')
else:
fh = sys.stdout
try:
yield fh
finally:
if fh is not sys.stdout:
fh.close()
Use it like this:
# For Python 2 you need this line
from __future__ import print_function
# writes to some_file
with smart_open('some_file') as fh:
print('some output', file=fh)
# writes to stdout
with smart_open() as fh:
print('some output', file=fh)
# writes to stdout
with smart_open('-') as fh:
print('some output', file=fh)
Stick with your current code. It’s simple and you can tell exactly what it’s doing just by glancing at it.
Another way would be with an inline if
:
handle = open(target, 'w') if target else sys.stdout
handle.write(content)
if handle is not sys.stdout:
handle.close()
But that isn’t much shorter than what you have and it looks arguably worse.
You could also make sys.stdout
unclosable, but that doesn’t seem too Pythonic:
sys.stdout.close = lambda: None
with (open(target, 'w') if target else sys.stdout) as handle:
handle.write(content)
I’d also go for a simple wrapper function, which can be pretty simple if you can ignore the mode (and consequently stdin vs. stdout), for example:
from contextlib import contextmanager
import sys
@contextmanager
def open_or_stdout(filename):
if filename != '-':
with open(filename, 'w') as f:
yield f
else:
yield sys.stdout
If you really must insist on something more “elegant”, i.e. a one-liner:
>>> import sys
>>> target = "foo.txt"
>>> content = "foo"
>>> (lambda target, content: (lambda target, content: filter(lambda h: not h.write(content), (target,))[0].close())(open(target, 'w'), content) if target else sys.stdout.write(content))(target, content)
foo.txt
appears and contains the text foo
.
Okay, if we are getting into one-liner wars, here’s:
(target and open(target, 'w') or sys.stdout).write(content)
I like Jacob’s original example as long as context is only written in one place. It would be a problem if you end up re-opening the file for many writes. I think I would just make the decision once at the top of the script and let the system close the file on exit:
output = target and open(target, 'w') or sys.stdout
...
output.write('thing onen')
...
output.write('thing twon')
You could include your own exit handler if you think its more tidy
import atexit
def cleanup_output():
global output
if output is not sys.stdout:
output.close()
atexit(cleanup_output)
Another possible solution: do not try to avoid the context manager exit method, just duplicate stdout.
with (os.fdopen(os.dup(sys.stdout.fileno()), 'w')
if target == '-'
else open(target, 'w')) as f:
f.write("Foo")
How about opening a new fd for sys.stdout? This way you won’t have any problems closing it:
if not target:
target = "/dev/stdout"
with open(target, 'w') as f:
f.write(content)
if (out != sys.stdout):
with open(out, 'wb') as f:
f.write(data)
else:
out.write(data)
Slight improvement in some cases.
An improvement of Wolph’s answer
import sys
import contextlib
@contextlib.contextmanager
def smart_open(filename: str, mode: str = 'r', *args, **kwargs):
'''Open files and i/o streams transparently.'''
if filename == '-':
if 'r' in mode:
stream = sys.stdin
else:
stream = sys.stdout
if 'b' in mode:
fh = stream.buffer # type: IO
else:
fh = stream
close = False
else:
fh = open(filename, mode, *args, **kwargs)
close = True
try:
yield fh
finally:
if close:
try:
fh.close()
except AttributeError:
pass
This allows binary IO and pass eventual extraneous arguments to open
if filename
is indeed a file name.
import contextlib
import sys
with contextlib.ExitStack() as stack:
h = stack.enter_context(open(target, 'w')) if target else sys.stdout
h.write(content)
Just two extra lines if you’re using Python 3.3 or higher: one line for the extra import
and one line for the stack.enter_context
.
If it’s fine that sys.stdout
is closed after with
body, you can also use patterns like this:
# Use stdout when target is "-"
with open(target, "w") if target != "-" else sys.stdout as f:
f.write("hello world")
# Use stdout when target is falsy (None, empty string, ...)
with open(target, "w") if target else sys.stdout as f:
f.write("hello world")
or even more generally:
with target if isinstance(target, io.IOBase) else open(target, "w") as f:
f.write("hello world")
The following solution is not a beauty, but from a time long, long ago; just before with …
handler = open(path, mode = 'a') if path else sys.stdout
try:
print('stuff', file = handler)
... # other stuff or more writes/prints, etc.
except Exception as e:
if not (path is None): handler.close()
raise e
handler.close()
One way to solve it is with polymorphism. Pathlib.path
has an open
method that functions as you would expect:
from pathlib import Path
output = Path("/path/to/file.csv")
with output.open(mode="w", encoding="utf-8") as f:
print("hello world", file=f)
we can copy this interface for printing
import sys
class Stdout:
def __init__(self, *args):
pass
def open(self, mode=None, encoding=None):
return self
def __enter__(self):
return sys.stdout
def __exit__(self, exc_type, exc_value, traceback):
pass
Now we simply replace Path
with Stdout
output = Stdout("/path/to/file.csv")
with output.open(mode="w", encoding="utf-8") as f:
print("hello world", file=f)
This isn’t necessarily better than overloading open
, but it’s a convenient solution if you’re using Path
objects.
With python 3 you can used wrap stdout
file descriptor with IO object and avoid closing on context leave it with closefd=False
:
h = open(target, 'w') if target else open(sys.stdout.fileno(), 'w', closefd=False)
with h as h:
h.write(content)
As pointed in Conditional with statement in Python, Python 3.7 allows using contextlib.nullcontext for that:
from contextlib import nullcontext
with open(target, "w") if target else nullcontext(sys.stdout) as f:
f.write(content)
This is a simpler and shorter version of the accepted answer
import contextlib, sys
def writer(fn):
@contextlib.contextmanager
def stdout():
yield sys.stdout
return open(fn, 'w') if fn else stdout()
usage:
with writer('') as w:
w.write('hellon')
with writer('file.txt') as w:
w.write('hellon')