Write file with specific permissions in Python
Question:
I’m trying to create a file that is only user-readable and -writable (0600).
Is the only way to do so by using os.open() as follows?
import os
fd = os.open('/path/to/file', os.O_WRONLY, 0o600)
myFileObject = os.fdopen(fd)
myFileObject.write(...)
myFileObject.close()
Ideally, I’d like to be able to use the with keyword so I can close the object automatically. Is there a better way to do what I’m doing above?
Answers:
update
Folks, while I thank you for the upvotes here, I myself have to argue against my originally proposed solution below. The reason is doing things this way, there will be an amount of time, however small, where the file does exist, and does not have the proper permissions in place – this leave open wide ways of attack, and even buggy behavior.
Of course creating the file with the correct permissions in the first place is the way to go – against the correctness of that, using Python’s with is just some candy.
So please, take this answer as an example of “what not to do”;
original post
You can use os.chmod instead:
>>> import os
>>> name = "eek.txt"
>>> with open(name, "wt") as myfile:
... os.chmod(name, 0o600)
... myfile.write("eeek")
...
>>> os.system("ls -lh " + name)
-rw------- 1 gwidion gwidion 4 2011-04-11 13:47 eek.txt
0
>>>
(Note that the way to use octals in Python is by being explicit – by prefixing it with “0o” like in “0o600“. In Python 2.x it would work writing just 0600 – but that is both misleading and deprecated.)
However, if your security is critical, you probably should resort to creating it with os.open, as you do and use os.fdopen to retrieve a Python File object from the file descriptor returned by os.open.
What’s the problem? file.close() will close the file even though it was open with os.open().
with os.fdopen(os.open('/path/to/file', os.O_WRONLY | os.O_CREAT, 0o600), 'w') as handle:
handle.write(...)
This answer addresses multiple concerns with the answer by vartec, especially the umask concern.
import os
import stat
# Define file params
fname = '/tmp/myfile'
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL # Refer to "man 2 open".
mode = stat.S_IRUSR | stat.S_IWUSR # This is 0o600.
umask = 0o777 ^ mode # Prevents always downgrading umask to 0.
# For security, remove file with potentially elevated mode
try:
os.remove(fname)
except OSError:
pass
# Open file descriptor
umask_original = os.umask(umask)
try:
fdesc = os.open(fname, flags, mode)
finally:
os.umask(umask_original)
# Open file handle and write to file
with os.fdopen(fdesc, 'w') as fout:
fout.write('somethingn')
If the desired mode is 0600, it can more clearly be specified as the octal number 0o600. Even better, just use the stat module.
Even though the old file is first deleted, a race condition is still possible. Including os.O_EXCL with os.O_CREAT in the flags will prevent the file from being created if it exists due to a race condition. This is a necessary secondary security measure to prevent opening a file that may already exist with a potentially elevated mode. In Python 3, FileExistsError with [Errno 17] is raised if the file exists.
Failing to first set the umask to 0 or to 0o777 ^ mode can lead to an incorrect mode (permission) being set by os.open. This is because the default umask is usually not 0, and it will be applied to the specified mode. For example, if my original umask is 2 i.e. 0o002, and my specified mode is 0o222, if I fail to first set the umask, the resulting file can instead have a mode of 0o220, which is not what I wanted. Per man 2 open, the mode of the created file is mode & ~umask.
The umask is restored to its original value as soon as possible. This getting and setting is not thread safe, and a threading.Lock must be used in a multithreaded application.
For more info about umask, refer to this thread.
I would like to suggest a modification of A-B-B’s excellent answer that separates the concerns a bit more clearly. The main advantage would be that you can handle exceptions that occur during opening the file descriptor separately from other problems during actual writing to the file.
The outer try ... finally block takes care of handling the permission and umask issues while opening the file descriptor. The inner with block deals with possible exceptions while working with the Python file object (as this was the OP’s wish):
try:
oldumask = os.umask(0)
fdesc = os.open(outfname, os.O_WRONLY | os.O_CREAT, 0o600)
with os.fdopen(fdesc, "w") as outf:
# ...write to outf, closes on success or on exceptions automatically...
except IOError, ... :
# ...handle possible os.open() errors here...
finally:
os.umask(oldumask)
If you want to append to the file instead of writing, then the file descriptor should be opened like this:
fdesc = os.open(outfname, os.O_WRONLY | os.O_CREAT | os.O_APPEND, 0o600)
and the file object like this:
with os.fdopen(fdesc, "a") as outf:
Of course all other usual combinations are possible.
The question is about setting the permissions to be sure the file will not be world-readable (only read/write for the current user).
Unfortunately, on its own, the code:
fd = os.open('/path/to/file', os.O_WRONLY, 0o600)
does not guarantee that permissions will be denied to the world. It does try to set r/w for the current user (provided that umask allows it), that’s it!
On two very different test systems, this code creates a file with -rw-r–r– with my default umask, and -rw-rw-rw- with umask(0) which is definitely not what is desired (and poses a serious security risk).
If you want to make sure that the file has no bits set for group and world, you have to umask these bits first (remember – umask is denial of permissions):
os.umask(0o177)
Besides, to be 100% sure that the file doesn’t already exist with different permissions, you have to chmod/delete it first (delete is safer, since you may not have write permissions in the target directory – and if you have security concerns, you don’t want to write some file where you’re not allowed to!), otherwise you may have a security issue if a hacker created the file before you with world-wide r/w permissions in anticipation of your move. In that case, os.open will open the file without setting its permissions at all and you’re left with a world r/w secret file…
So you need:
import os
if os.path.isfile(file):
os.remove(file)
original_umask = os.umask(0o177) # 0o777 ^ 0o600
try:
handle = os.fdopen(os.open(file, os.O_WRONLY | os.O_CREAT, 0o600), 'w')
finally:
os.umask(original_umask)
This is the safe way to ensure the creation of a -rw——- file regardless of your environment and configuration. And of course you can catch and deal with the IOErrors as needed. If you don’t have write permissions in the target directory, you shouldn’t be able to create the file, and if it already existed the delete will fail.
I’d do differently.
from contextlib import contextmanager
@contextmanager
def umask_helper(desired_umask):
""" A little helper to safely set and restore umask(2). """
try:
prev_umask = os.umask(desired_umask)
yield
finally:
os.umask(prev_umask)
# ---------------------------------- […] ---------------------------------- #
[…]
with umask_helper(0o077):
os.mkdir(os.path.dirname(MY_FILE))
with open(MY_FILE, 'wt') as f:
[…]
File-manipulating code tends to be already try–except-heavy; making it even worse with os.umask’s finally isn’t going to bring your eyes any more joy. Meanwhile, rolling your own context manager is that easy, and results in somewhat neater indentation nesting.
I’m trying to create a file that is only user-readable and -writable (0600).
Is the only way to do so by using os.open() as follows?
import os
fd = os.open('/path/to/file', os.O_WRONLY, 0o600)
myFileObject = os.fdopen(fd)
myFileObject.write(...)
myFileObject.close()
Ideally, I’d like to be able to use the with keyword so I can close the object automatically. Is there a better way to do what I’m doing above?
update
Folks, while I thank you for the upvotes here, I myself have to argue against my originally proposed solution below. The reason is doing things this way, there will be an amount of time, however small, where the file does exist, and does not have the proper permissions in place – this leave open wide ways of attack, and even buggy behavior.
Of course creating the file with the correct permissions in the first place is the way to go – against the correctness of that, using Python’s with is just some candy.
So please, take this answer as an example of “what not to do”;
original post
You can use os.chmod instead:
>>> import os
>>> name = "eek.txt"
>>> with open(name, "wt") as myfile:
... os.chmod(name, 0o600)
... myfile.write("eeek")
...
>>> os.system("ls -lh " + name)
-rw------- 1 gwidion gwidion 4 2011-04-11 13:47 eek.txt
0
>>>
(Note that the way to use octals in Python is by being explicit – by prefixing it with “0o” like in “0o600“. In Python 2.x it would work writing just 0600 – but that is both misleading and deprecated.)
However, if your security is critical, you probably should resort to creating it with os.open, as you do and use os.fdopen to retrieve a Python File object from the file descriptor returned by os.open.
What’s the problem? file.close() will close the file even though it was open with os.open().
with os.fdopen(os.open('/path/to/file', os.O_WRONLY | os.O_CREAT, 0o600), 'w') as handle:
handle.write(...)
This answer addresses multiple concerns with the answer by vartec, especially the umask concern.
import os
import stat
# Define file params
fname = '/tmp/myfile'
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL # Refer to "man 2 open".
mode = stat.S_IRUSR | stat.S_IWUSR # This is 0o600.
umask = 0o777 ^ mode # Prevents always downgrading umask to 0.
# For security, remove file with potentially elevated mode
try:
os.remove(fname)
except OSError:
pass
# Open file descriptor
umask_original = os.umask(umask)
try:
fdesc = os.open(fname, flags, mode)
finally:
os.umask(umask_original)
# Open file handle and write to file
with os.fdopen(fdesc, 'w') as fout:
fout.write('somethingn')
If the desired mode is 0600, it can more clearly be specified as the octal number 0o600. Even better, just use the stat module.
Even though the old file is first deleted, a race condition is still possible. Including os.O_EXCL with os.O_CREAT in the flags will prevent the file from being created if it exists due to a race condition. This is a necessary secondary security measure to prevent opening a file that may already exist with a potentially elevated mode. In Python 3, FileExistsError with [Errno 17] is raised if the file exists.
Failing to first set the umask to 0 or to 0o777 ^ mode can lead to an incorrect mode (permission) being set by os.open. This is because the default umask is usually not 0, and it will be applied to the specified mode. For example, if my original umask is 2 i.e. 0o002, and my specified mode is 0o222, if I fail to first set the umask, the resulting file can instead have a mode of 0o220, which is not what I wanted. Per man 2 open, the mode of the created file is mode & ~umask.
The umask is restored to its original value as soon as possible. This getting and setting is not thread safe, and a threading.Lock must be used in a multithreaded application.
For more info about umask, refer to this thread.
I would like to suggest a modification of A-B-B’s excellent answer that separates the concerns a bit more clearly. The main advantage would be that you can handle exceptions that occur during opening the file descriptor separately from other problems during actual writing to the file.
The outer try ... finally block takes care of handling the permission and umask issues while opening the file descriptor. The inner with block deals with possible exceptions while working with the Python file object (as this was the OP’s wish):
try:
oldumask = os.umask(0)
fdesc = os.open(outfname, os.O_WRONLY | os.O_CREAT, 0o600)
with os.fdopen(fdesc, "w") as outf:
# ...write to outf, closes on success or on exceptions automatically...
except IOError, ... :
# ...handle possible os.open() errors here...
finally:
os.umask(oldumask)
If you want to append to the file instead of writing, then the file descriptor should be opened like this:
fdesc = os.open(outfname, os.O_WRONLY | os.O_CREAT | os.O_APPEND, 0o600)
and the file object like this:
with os.fdopen(fdesc, "a") as outf:
Of course all other usual combinations are possible.
The question is about setting the permissions to be sure the file will not be world-readable (only read/write for the current user).
Unfortunately, on its own, the code:
fd = os.open('/path/to/file', os.O_WRONLY, 0o600)
does not guarantee that permissions will be denied to the world. It does try to set r/w for the current user (provided that umask allows it), that’s it!
On two very different test systems, this code creates a file with -rw-r–r– with my default umask, and -rw-rw-rw- with umask(0) which is definitely not what is desired (and poses a serious security risk).
If you want to make sure that the file has no bits set for group and world, you have to umask these bits first (remember – umask is denial of permissions):
os.umask(0o177)
Besides, to be 100% sure that the file doesn’t already exist with different permissions, you have to chmod/delete it first (delete is safer, since you may not have write permissions in the target directory – and if you have security concerns, you don’t want to write some file where you’re not allowed to!), otherwise you may have a security issue if a hacker created the file before you with world-wide r/w permissions in anticipation of your move. In that case, os.open will open the file without setting its permissions at all and you’re left with a world r/w secret file…
So you need:
import os
if os.path.isfile(file):
os.remove(file)
original_umask = os.umask(0o177) # 0o777 ^ 0o600
try:
handle = os.fdopen(os.open(file, os.O_WRONLY | os.O_CREAT, 0o600), 'w')
finally:
os.umask(original_umask)
This is the safe way to ensure the creation of a -rw——- file regardless of your environment and configuration. And of course you can catch and deal with the IOErrors as needed. If you don’t have write permissions in the target directory, you shouldn’t be able to create the file, and if it already existed the delete will fail.
I’d do differently.
from contextlib import contextmanager
@contextmanager
def umask_helper(desired_umask):
""" A little helper to safely set and restore umask(2). """
try:
prev_umask = os.umask(desired_umask)
yield
finally:
os.umask(prev_umask)
# ---------------------------------- […] ---------------------------------- #
[…]
with umask_helper(0o077):
os.mkdir(os.path.dirname(MY_FILE))
with open(MY_FILE, 'wt') as f:
[…]
File-manipulating code tends to be already try–except-heavy; making it even worse with os.umask’s finally isn’t going to bring your eyes any more joy. Meanwhile, rolling your own context manager is that easy, and results in somewhat neater indentation nesting.