Python os.path.join on Windows
Question:
I am trying to learn python and am making a program that will output a script. I want to use os.path.join, but am pretty confused. According to the docs if I say:
os.path.join('c:', 'sourcedir')
I get "C:sourcedir"
. According to the docs, this is normal, right?
But when I use the copytree command, Python will output it the desired way, for example:
import shutil
src = os.path.join('c:', 'src')
dst = os.path.join('c:', 'dst')
shutil.copytree(src, dst)
Here is the error code I get:
WindowsError: [Error 3] The system cannot find the path specified: 'C:src/*.*'
If I wrap the os.path.join
with os.path.normpath
I get the same error.
If this os.path.join
can’t be used this way, then I am confused as to its purpose.
According to the pages suggested by Stack Overflow, slashes should not be used in join—that is correct, I assume?
Answers:
to join a windows path, try
mypath=os.path.join('c:\', 'sourcedir')
basically, you will need to escape the slash
The reason os.path.join('C:', 'src')
is not working as you expect is because of something in the documentation that you linked to:
Note that on Windows, since there is a
current directory for each drive,
os.path.join(“c:”, “foo”) represents a
path relative to the current directory
on drive C: (c:foo), not c:foo.
As ghostdog said, you probably want mypath=os.path.join('c:\', 'sourcedir')
Windows has a concept of current directory for each drive. Because of that, "c:sourcedir"
means “sourcedir” inside the current C: directory, and you’ll need to specify an absolute directory.
Any of these should work and give the same result, but I don’t have a Windows VM fired up at the moment to double check:
"c:/sourcedir"
os.path.join("/", "c:", "sourcedir")
os.path.join("c:/", "sourcedir")
To be pedantic, it’s probably not good to hardcode either / or as the path separator. Maybe this would be best?
mypath = os.path.join('c:%s' % os.sep, 'sourcedir')
or
mypath = os.path.join('c:' + os.sep, 'sourcedir')
To be even more pedantic, the most python doc consistent answer would be:
mypath = os.path.join('c:', os.sep, 'sourcedir')
Since you also need os.sep for the posix root path:
mypath = os.path.join(os.sep, 'usr', 'lib')
I’d say this is a (windows)python bug.
Why bug?
I think this statement should be True
os.path.join(*os.path.dirname(os.path.abspath(__file__)).split(os.path.sep))==os.path.dirname(os.path.abspath(__file__))
But it is False
on windows machines.
Consent with @georg-
I would say then why we need lame os.path.join
– better to use str.join
or unicode.join
e.g.
sys.path.append('{0}'.join(os.path.dirname(__file__).split(os.path.sep)[0:-1]).format(os.path.sep))
You have a few possible approaches to treat path on Windows, from the most hardcoded ones (as using raw string literals or escaping backslashes) to the least ones. Here follows a few examples that will work as expected. Use what better fits your needs.
In[1]: from os.path import join, isdir
In[2]: from os import sep
In[3]: isdir(join("c:", "\", "Users"))
Out[3]: True
In[4]: isdir(join("c:", "/", "Users"))
Out[4]: True
In[5]: isdir(join("c:", sep, "Users"))
Out[5]: True
For a system-agnostic solution that works on both Windows and Linux, no matter what the input path, one could use
def joinpath(rootdir, targetdir):
return os.path.join(os.sep, rootdir + os.sep, targetdir)
On Windows:
>>> joinpath("C:", "Windows")
'C:\Windows'
>>> joinpath("C:\Windows", "src")
'C:\Windows\src'
On Linux:
>>> joinpath("usr", "lib")
'/usr/lib'
>>> joinpath("/usr/lib", "src")
'/usr/lib/src'
answering to your comment : “the others ‘//’ ‘c:’, ‘c:\’ did not work (C:\ created two backslashes, C: didn’t work at all)”
On windows using
os.path.join('c:', 'sourcedir')
will automatically add two backslashes \
in front of sourcedir.
To resolve the path, as python works on windows also with forward slashes -> ‘/’, simply add .replace('\','/')
with os.path.join
as below:-
os.path.join('c:\', 'sourcedir').replace('\','/')
e.g: os.path.join('c:\', 'temp').replace('\','/')
output : ‘C:/temp’
The proposed solutions are interesting and offer a good reference, however they are only partially satisfying. It is ok to manually add the separator when you have a single specific case or you know the format of the input string, but there can be cases where you want to do it programmatically on generic inputs.
With a bit of experimenting, I believe the criteria is that the path delimiter is not added if the first segment is a drive letter, meaning a single letter followed by a colon, no matter if it corresponds to a real unit.
For example:
import os
testval = ['c:','c:\','d:','j:','jr:','data:']
for t in testval:
print ('test value: ',t,', join to "folder"',os.path.join(t,'folder'))
test value: c: , join to "folder" c:folder
test value: c: , join to "folder" c:folder
test value: d: , join to "folder" d:folder
test value: j: , join to "folder" j:folder
test value: jr: , join to "folder" jr:folder
test value: data: , join to "folder" data:folder
A convenient way to test for the criteria and apply a path correction can be to use os.path.splitdrive
comparing the first returned element to the test value, like t+os.path.sep if os.path.splitdrive(t)[0]==t else t
.
Test:
for t in testval:
corrected = t+os.path.sep if os.path.splitdrive(t)[0]==t else t
print ('original: %stcorrected: %s'%(t,corrected),' join corrected->',os.path.join(corrected,'folder'))
original: c: corrected: c: join corrected-> c:folder
original: c: corrected: c: join corrected-> c:folder
original: d: corrected: d: join corrected-> d:folder
original: j: corrected: j: join corrected-> j:folder
original: jr: corrected: jr: join corrected-> jr:folder
original: data: corrected: data: join corrected-> data:folder
it can be probably be improved to be more robust for trailing spaces, and I have tested it only on windows, but I hope it gives an idea.
See also Os.path : can you explain this behavior? for interesting details on systems other then windows.
I got around this by using:
os.sep.join(list('C:', 'sourcedir'))
join
here is not from os.path.join()
, but from ''.join()
It can be useful in the following situation:
import os
some_path = r'C:some_foldersome_file.txt'
path_items = some_path.split(os.sep)
same_path = os.sep.join(path_items)
For simplicity’s sake, a workaround:
my_path = r'D:testtest2file.txt' # original path string
drive_letter = my_path.split(':')[0] # avoid os.path.join relative path bs
my_path_parts = os.path.normpath(my_path.split(':')[1]).split(os.sep)
# do some path cmponent modifications here if you want
if drive_letter: # if drive letter exists
drive_letter += ':\'
my_new_path = drive_letter + os.path.join(*my_path_parts)
my_new_path
In windows using os.paht.join("/", "Temp")
will result in /Temp by default but as strange as it sounds, there is no problem in using that path as a full path equivalent to "C:/Temp" and it works for both saving files and opening files.
I am trying to learn python and am making a program that will output a script. I want to use os.path.join, but am pretty confused. According to the docs if I say:
os.path.join('c:', 'sourcedir')
I get "C:sourcedir"
. According to the docs, this is normal, right?
But when I use the copytree command, Python will output it the desired way, for example:
import shutil
src = os.path.join('c:', 'src')
dst = os.path.join('c:', 'dst')
shutil.copytree(src, dst)
Here is the error code I get:
WindowsError: [Error 3] The system cannot find the path specified: 'C:src/*.*'
If I wrap the os.path.join
with os.path.normpath
I get the same error.
If this os.path.join
can’t be used this way, then I am confused as to its purpose.
According to the pages suggested by Stack Overflow, slashes should not be used in join—that is correct, I assume?
to join a windows path, try
mypath=os.path.join('c:\', 'sourcedir')
basically, you will need to escape the slash
The reason os.path.join('C:', 'src')
is not working as you expect is because of something in the documentation that you linked to:
Note that on Windows, since there is a
current directory for each drive,
os.path.join(“c:”, “foo”) represents a
path relative to the current directory
on drive C: (c:foo), not c:foo.
As ghostdog said, you probably want mypath=os.path.join('c:\', 'sourcedir')
Windows has a concept of current directory for each drive. Because of that, "c:sourcedir"
means “sourcedir” inside the current C: directory, and you’ll need to specify an absolute directory.
Any of these should work and give the same result, but I don’t have a Windows VM fired up at the moment to double check:
"c:/sourcedir"
os.path.join("/", "c:", "sourcedir")
os.path.join("c:/", "sourcedir")
To be pedantic, it’s probably not good to hardcode either / or as the path separator. Maybe this would be best?
mypath = os.path.join('c:%s' % os.sep, 'sourcedir')
or
mypath = os.path.join('c:' + os.sep, 'sourcedir')
To be even more pedantic, the most python doc consistent answer would be:
mypath = os.path.join('c:', os.sep, 'sourcedir')
Since you also need os.sep for the posix root path:
mypath = os.path.join(os.sep, 'usr', 'lib')
I’d say this is a (windows)python bug.
Why bug?
I think this statement should be True
os.path.join(*os.path.dirname(os.path.abspath(__file__)).split(os.path.sep))==os.path.dirname(os.path.abspath(__file__))
But it is False
on windows machines.
Consent with @georg-
I would say then why we need lame os.path.join
– better to use str.join
or unicode.join
e.g.
sys.path.append('{0}'.join(os.path.dirname(__file__).split(os.path.sep)[0:-1]).format(os.path.sep))
You have a few possible approaches to treat path on Windows, from the most hardcoded ones (as using raw string literals or escaping backslashes) to the least ones. Here follows a few examples that will work as expected. Use what better fits your needs.
In[1]: from os.path import join, isdir
In[2]: from os import sep
In[3]: isdir(join("c:", "\", "Users"))
Out[3]: True
In[4]: isdir(join("c:", "/", "Users"))
Out[4]: True
In[5]: isdir(join("c:", sep, "Users"))
Out[5]: True
For a system-agnostic solution that works on both Windows and Linux, no matter what the input path, one could use
def joinpath(rootdir, targetdir):
return os.path.join(os.sep, rootdir + os.sep, targetdir)
On Windows:
>>> joinpath("C:", "Windows")
'C:\Windows'
>>> joinpath("C:\Windows", "src")
'C:\Windows\src'
On Linux:
>>> joinpath("usr", "lib")
'/usr/lib'
>>> joinpath("/usr/lib", "src")
'/usr/lib/src'
answering to your comment : “the others ‘//’ ‘c:’, ‘c:\’ did not work (C:\ created two backslashes, C: didn’t work at all)”
On windows using
os.path.join('c:', 'sourcedir')
will automatically add two backslashes \
in front of sourcedir.
To resolve the path, as python works on windows also with forward slashes -> ‘/’, simply add .replace('\','/')
with os.path.join
as below:-
os.path.join('c:\', 'sourcedir').replace('\','/')
e.g: os.path.join('c:\', 'temp').replace('\','/')
output : ‘C:/temp’
The proposed solutions are interesting and offer a good reference, however they are only partially satisfying. It is ok to manually add the separator when you have a single specific case or you know the format of the input string, but there can be cases where you want to do it programmatically on generic inputs.
With a bit of experimenting, I believe the criteria is that the path delimiter is not added if the first segment is a drive letter, meaning a single letter followed by a colon, no matter if it corresponds to a real unit.
For example:
import os
testval = ['c:','c:\','d:','j:','jr:','data:']
for t in testval:
print ('test value: ',t,', join to "folder"',os.path.join(t,'folder'))
test value: c: , join to "folder" c:folder test value: c: , join to "folder" c:folder test value: d: , join to "folder" d:folder test value: j: , join to "folder" j:folder test value: jr: , join to "folder" jr:folder test value: data: , join to "folder" data:folder
A convenient way to test for the criteria and apply a path correction can be to use os.path.splitdrive
comparing the first returned element to the test value, like t+os.path.sep if os.path.splitdrive(t)[0]==t else t
.
Test:
for t in testval:
corrected = t+os.path.sep if os.path.splitdrive(t)[0]==t else t
print ('original: %stcorrected: %s'%(t,corrected),' join corrected->',os.path.join(corrected,'folder'))
original: c: corrected: c: join corrected-> c:folder original: c: corrected: c: join corrected-> c:folder original: d: corrected: d: join corrected-> d:folder original: j: corrected: j: join corrected-> j:folder original: jr: corrected: jr: join corrected-> jr:folder original: data: corrected: data: join corrected-> data:folder
it can be probably be improved to be more robust for trailing spaces, and I have tested it only on windows, but I hope it gives an idea.
See also Os.path : can you explain this behavior? for interesting details on systems other then windows.
I got around this by using:
os.sep.join(list('C:', 'sourcedir'))
join
here is not from os.path.join()
, but from ''.join()
It can be useful in the following situation:
import os
some_path = r'C:some_foldersome_file.txt'
path_items = some_path.split(os.sep)
same_path = os.sep.join(path_items)
For simplicity’s sake, a workaround:
my_path = r'D:testtest2file.txt' # original path string
drive_letter = my_path.split(':')[0] # avoid os.path.join relative path bs
my_path_parts = os.path.normpath(my_path.split(':')[1]).split(os.sep)
# do some path cmponent modifications here if you want
if drive_letter: # if drive letter exists
drive_letter += ':\'
my_new_path = drive_letter + os.path.join(*my_path_parts)
my_new_path
In windows using os.paht.join("/", "Temp")
will result in /Temp by default but as strange as it sounds, there is no problem in using that path as a full path equivalent to "C:/Temp" and it works for both saving files and opening files.