open read and close a file in 1 line of code
Question:
Now I use:
pageHeadSectionFile = open('pagehead.section.htm','r')
output = pageHeadSectionFile.read()
pageHeadSectionFile.close()
But to make the code look better, I can do:
output = open('pagehead.section.htm','r').read()
When using the above syntax, how do I close the file to free up system resources?
Answers:
You don’t really have to close it – Python will do it automatically either during garbage collection or at program exit. But as @delnan noted, it’s better practice to explicitly close it for various reasons.
So, what you can do to keep it short, simple and explicit:
with open('pagehead.section.htm', 'r') as f:
output = f.read()
Now it’s just two lines and pretty readable, I think.
Using CPython, your file will be closed immediately after the line is executed, because the file object is immediately garbage collected. There are two drawbacks, though:
-
In Python implementations different from CPython, the file often isn’t immediately closed, but rather at a later time, beyond your control.
-
In Python 3.2 or above, this will throw a ResourceWarning
, if enabled.
Better to invest one additional line:
with open('pagehead.section.htm','r') as f:
output = f.read()
This will ensure that the file is correctly closed under all circumstances.
What you can do is to use the with
statement, and write the two steps on one line:
>>> with open('pagehead.section.htm', 'r') as fin: output = fin.read();
>>> print(output)
some content
The with
statement will take care to call __exit__
function of the given object even if something bad happened in your code; it’s close to the try... finally
syntax. For object returned by open
, __exit__
corresponds to file closure.
This statement has been introduced with Python 2.6.
I frequently do something like this when I need to get a few lines surrounding something I’ve grepped in a log file:
$ grep -n "xlrd" requirements.txt | awk -F ":" '{print $1}'
54
$ python -c "with open('requirements.txt') as file: print ''.join(file.readlines()[52:55])"
wsgiref==0.1.2
xlrd==0.9.2
xlwt==0.7.5
use ilio: (inline io):
just one function call instead of file open(), read(), close().
from ilio import read
content = read('filename')
Python Standard Library Pathlib module does what you looking for:
Path('pagehead.section.htm').read_text()
Don’t forget to import Path:
jsk@dev1:~$ python3
Python 3.5.2 (default, Sep 10 2016, 08:21:44)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> (Path("/etc") / "hostname").read_text()
'dev1.examplen'
with open('pagehead.section.htm')as f_contents=f.read()
Using more_itertools.with_iter
, it is possible to open, read, close and assign an equivalent output
in one line (excluding the import statement):
import more_itertools as mit
output = "".join(line for line in mit.with_iter(open("pagehead.section.htm", "r")))
Although possible, I would look for another approach other than assigning the contents of a file to a variable, i.e. lazy iteration – this can be done using a traditional with
block or in the example above by removing join()
and iterating output
.
No need to import any special libraries to do this.
Use normal syntax and it will open the file for reading then close it.
with open("/etc/hostname","r") as f: print f.read()
or
with open("/etc/hosts","r") as f: x = f.read().splitlines()
which gives you an array x containing the lines, and can be printed like so:
for line in x: print line
These one-liners are very helpful for maintenance – basically self-documenting.
If you want that warm and fuzzy feeling just go with with.
For python 3.6 I ran these two programs under a fresh start of IDLE, giving runtimes of:
0.002000093460083008 Test A
0.0020003318786621094 Test B: with guaranteed close
So not much of a difference.
#--------*---------*---------*---------*---------*---------*---------*---------*
# Desc: Test A for reading a text file line-by-line into a list
#--------*---------*---------*---------*---------*---------*---------*---------*
import sys
import time
# # MAINLINE
if __name__ == '__main__':
print("OK, starting program...")
inTextFile = '/Users/Mike/Desktop/garbage.txt'
# # Test: A: no 'with;
c=[]
start_time = time.time()
c = open(inTextFile).read().splitlines()
print("--- %s seconds ---" % (time.time() - start_time))
print("OK, program execution has ended.")
sys.exit() # END MAINLINE
OUTPUT:
OK, starting program...
--- 0.002000093460083008 seconds ---
OK, program execution has ended.
#--------*---------*---------*---------*---------*---------*---------*---------*
# Desc: Test B for reading a text file line-by-line into a list
#--------*---------*---------*---------*---------*---------*---------*---------*
import sys
import time
# # MAINLINE
if __name__ == '__main__':
print("OK, starting program...")
inTextFile = '/Users/Mike/Desktop/garbage.txt'
# # Test: B: using 'with'
c=[]
start_time = time.time()
with open(inTextFile) as D: c = D.read().splitlines()
print("--- %s seconds ---" % (time.time() - start_time))
print("OK, program execution has ended.")
sys.exit() # END MAINLINE
OUTPUT:
OK, starting program...
--- 0.0020003318786621094 seconds ---
OK, program execution has ended.
I think the most natural way for achieving this is to define a function.
def read(filename):
f = open(filename, 'r')
output = f.read()
f.close()
return output
Then you can do the following:
output = read('pagehead.section.htm')
Now I use:
pageHeadSectionFile = open('pagehead.section.htm','r')
output = pageHeadSectionFile.read()
pageHeadSectionFile.close()
But to make the code look better, I can do:
output = open('pagehead.section.htm','r').read()
When using the above syntax, how do I close the file to free up system resources?
You don’t really have to close it – Python will do it automatically either during garbage collection or at program exit. But as @delnan noted, it’s better practice to explicitly close it for various reasons.
So, what you can do to keep it short, simple and explicit:
with open('pagehead.section.htm', 'r') as f:
output = f.read()
Now it’s just two lines and pretty readable, I think.
Using CPython, your file will be closed immediately after the line is executed, because the file object is immediately garbage collected. There are two drawbacks, though:
-
In Python implementations different from CPython, the file often isn’t immediately closed, but rather at a later time, beyond your control.
-
In Python 3.2 or above, this will throw a
ResourceWarning
, if enabled.
Better to invest one additional line:
with open('pagehead.section.htm','r') as f:
output = f.read()
This will ensure that the file is correctly closed under all circumstances.
What you can do is to use the with
statement, and write the two steps on one line:
>>> with open('pagehead.section.htm', 'r') as fin: output = fin.read();
>>> print(output)
some content
The with
statement will take care to call __exit__
function of the given object even if something bad happened in your code; it’s close to the try... finally
syntax. For object returned by open
, __exit__
corresponds to file closure.
This statement has been introduced with Python 2.6.
I frequently do something like this when I need to get a few lines surrounding something I’ve grepped in a log file:
$ grep -n "xlrd" requirements.txt | awk -F ":" '{print $1}'
54
$ python -c "with open('requirements.txt') as file: print ''.join(file.readlines()[52:55])"
wsgiref==0.1.2
xlrd==0.9.2
xlwt==0.7.5
use ilio: (inline io):
just one function call instead of file open(), read(), close().
from ilio import read
content = read('filename')
Python Standard Library Pathlib module does what you looking for:
Path('pagehead.section.htm').read_text()
Don’t forget to import Path:
jsk@dev1:~$ python3
Python 3.5.2 (default, Sep 10 2016, 08:21:44)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> (Path("/etc") / "hostname").read_text()
'dev1.examplen'
with open('pagehead.section.htm')as f_contents=f.read()
Using more_itertools.with_iter
, it is possible to open, read, close and assign an equivalent output
in one line (excluding the import statement):
import more_itertools as mit
output = "".join(line for line in mit.with_iter(open("pagehead.section.htm", "r")))
Although possible, I would look for another approach other than assigning the contents of a file to a variable, i.e. lazy iteration – this can be done using a traditional with
block or in the example above by removing join()
and iterating output
.
No need to import any special libraries to do this.
Use normal syntax and it will open the file for reading then close it.
with open("/etc/hostname","r") as f: print f.read()
or
with open("/etc/hosts","r") as f: x = f.read().splitlines()
which gives you an array x containing the lines, and can be printed like so:
for line in x: print line
These one-liners are very helpful for maintenance – basically self-documenting.
If you want that warm and fuzzy feeling just go with with.
For python 3.6 I ran these two programs under a fresh start of IDLE, giving runtimes of:
0.002000093460083008 Test A
0.0020003318786621094 Test B: with guaranteed close
So not much of a difference.
#--------*---------*---------*---------*---------*---------*---------*---------*
# Desc: Test A for reading a text file line-by-line into a list
#--------*---------*---------*---------*---------*---------*---------*---------*
import sys
import time
# # MAINLINE
if __name__ == '__main__':
print("OK, starting program...")
inTextFile = '/Users/Mike/Desktop/garbage.txt'
# # Test: A: no 'with;
c=[]
start_time = time.time()
c = open(inTextFile).read().splitlines()
print("--- %s seconds ---" % (time.time() - start_time))
print("OK, program execution has ended.")
sys.exit() # END MAINLINE
OUTPUT:
OK, starting program...
--- 0.002000093460083008 seconds ---
OK, program execution has ended.
#--------*---------*---------*---------*---------*---------*---------*---------*
# Desc: Test B for reading a text file line-by-line into a list
#--------*---------*---------*---------*---------*---------*---------*---------*
import sys
import time
# # MAINLINE
if __name__ == '__main__':
print("OK, starting program...")
inTextFile = '/Users/Mike/Desktop/garbage.txt'
# # Test: B: using 'with'
c=[]
start_time = time.time()
with open(inTextFile) as D: c = D.read().splitlines()
print("--- %s seconds ---" % (time.time() - start_time))
print("OK, program execution has ended.")
sys.exit() # END MAINLINE
OUTPUT:
OK, starting program...
--- 0.0020003318786621094 seconds ---
OK, program execution has ended.
I think the most natural way for achieving this is to define a function.
def read(filename):
f = open(filename, 'r')
output = f.read()
f.close()
return output
Then you can do the following:
output = read('pagehead.section.htm')