Is module __file__ attribute absolute or relative?
Question:
I’m having trouble understanding __file__. From what I understand, __file__ returns the absolute path from which the module was loaded.
I’m having problem producing this: I have a abc.py with one statement print __file__, running from /d/projects/ python abc.py returns abc.py. running from /d/ returns projects/abc.py. Any reasons why?
Answers:
From the documentation:
__file__ is the pathname of the file from which the module was loaded, if it was loaded from a file. The __file__ attribute is not present for C modules that are statically linked into the interpreter; for extension modules loaded dynamically from a shared library, it is the pathname of the shared library file.
From the mailing list thread linked by @kindall in a comment to the question:
I haven’t tried to repro this particular example, but the reason is
that we don’t want to have to call getpwd() on every import nor do we
want to have some kind of in-process variable to cache the current
directory. (getpwd() is relatively slow and can sometimes fail
outright, and trying to cache it has a certain risk of being wrong.)
What we do instead, is code in site.py that walks over the elements of
sys.path and turns them into absolute paths. However this code runs
before ” is inserted in the front of sys.path, so that the initial
value of sys.path is ”.
For the rest of this, consider sys.path not to include ''.
So, if you are outside the part of sys.path that contains the module, you’ll get an absolute path. If you are inside the part of sys.path that contains the module, you’ll get a relative path.
If you load a module in the current directory, and the current directory isn’t in sys.path, you’ll get an absolute path.
If you load a module in the current directory, and the current directory is in sys.path, you’ll get a relative path.
With the help of the of Guido mail provided by @kindall, we can understand the standard import process as trying to find the module in each member of sys.path, and file as the result of this lookup (more details in PyMOTW Modules and Imports.). So if the module is located in an absolute path in sys.path the result is absolute, but if it is located in a relative path in sys.path the result is relative.
Now the site.py startup file takes care of delivering only absolute path in sys.path, except the initial '', so if you don’t change it by other means than setting the PYTHONPATH (whose path are also made absolute, before prefixing sys.path), you will get always an absolute path, but when the module is accessed through the current directory.
Now if you trick sys.path in a funny way you can get anything.
As example if you have a sample module foo.py in /tmp/ with the code:
import sys
print(sys.path)
print (__file__)
If you go in /tmp you get:
>>> import foo
['', '/tmp', '/usr/lib/python3.3', ...]
./foo.py
When in in /home/user, if you add /tmp your PYTHONPATH you get:
>>> import foo
['', '/tmp', '/usr/lib/python3.3', ...]
/tmp/foo.py
Even if you add ../../tmp, it will be normalized and the result is the same.
But if instead of using PYTHONPATH you use directly some funny path
you get a result as funny as the cause.
>>> import sys
>>> sys.path.append('../../tmp')
>>> import foo
['', '/usr/lib/python3.3', .... , '../../tmp']
../../tmp/foo.py
Guido explains in the above cited thread, why python do not try to transform all entries in absolute paths:
we don’t want to have to call getpwd() on every import ….
getpwd() is relatively slow and can sometimes fail outright,
So your path is used as it is.
__file__ is absolute since Python 3.4, except when executing a script directly using a relative path:
Module __file__ attributes (and related values) should now always contain absolute paths by default, with the sole exception of __main__.__file__ when a script has been executed directly using a relative path. (Contributed by Brett Cannon in bpo-18416.)
Not sure if it resolves symlinks though.
Example of passing a relative path:
$ python script.py
Late simple example:
from os import path, getcwd, chdir
def print_my_path():
print('cwd: {}'.format(getcwd()))
print('__file__:{}'.format(__file__))
print('abspath: {}'.format(path.abspath(__file__)))
print_my_path()
chdir('..')
print_my_path()
Under Python-2.*, the second call incorrectly determines the path.abspath(__file__) based on the current directory:
cwd: C:codespy
__file__:cwd_mayhem.py
abspath: C:codespycwd_mayhem.py
cwd: C:codes
__file__:cwd_mayhem.py
abspath: C:codescwd_mayhem.py
As noted by @techtonik, in Python 3.4+, this will work fine since __file__ returns an absolute path.
__file__ can be relative or absolute depending on the python version used and whether the module is executed directly or not.
TL;DR:
- From python 3.5 to 3.8,
__file__ is set to the relative path of the module w.r.t. the current working directory if the module is called directly. Otherwise it is set to the absolute path.
- Since python 3.9,
__file__ is set to the absolute path even if the corresponding module is executed directly.
For instance, having the following setup (inspired by this answer):
# x.py:
from pathlib import Path
import y
print(__file__)
print(Path(__file__))
print(Path(__file__).resolve())
# y.py:
from pathlib import Path
print(__file__)
print(Path(__file__))
with x.py and y.py in the same directory. The different outputs after going to this directory and executing:
python x.py
are:
Python 3.5 – 3.8:
D:py_testsy.py
D:py_testsy.py
x.py
x.py
D:py_testsx.py
Python 3.9 – 3.11
D:py_testsy.py
D:py_testsy.py
D:py_testsx.py
D:py_testsx.py
D:py_testsx.py
I’m having trouble understanding __file__. From what I understand, __file__ returns the absolute path from which the module was loaded.
I’m having problem producing this: I have a abc.py with one statement print __file__, running from /d/projects/ python abc.py returns abc.py. running from /d/ returns projects/abc.py. Any reasons why?
From the documentation:
__file__is the pathname of the file from which the module was loaded, if it was loaded from a file. The__file__attribute is not present for C modules that are statically linked into the interpreter; for extension modules loaded dynamically from a shared library, it is the pathname of the shared library file.
From the mailing list thread linked by @kindall in a comment to the question:
I haven’t tried to repro this particular example, but the reason is
that we don’t want to have to call getpwd() on every import nor do we
want to have some kind of in-process variable to cache the current
directory. (getpwd() is relatively slow and can sometimes fail
outright, and trying to cache it has a certain risk of being wrong.)What we do instead, is code in site.py that walks over the elements of
sys.path and turns them into absolute paths. However this code runs
before ” is inserted in the front of sys.path, so that the initial
value of sys.path is ”.
For the rest of this, consider sys.path not to include ''.
So, if you are outside the part of sys.path that contains the module, you’ll get an absolute path. If you are inside the part of sys.path that contains the module, you’ll get a relative path.
If you load a module in the current directory, and the current directory isn’t in sys.path, you’ll get an absolute path.
If you load a module in the current directory, and the current directory is in sys.path, you’ll get a relative path.
With the help of the of Guido mail provided by @kindall, we can understand the standard import process as trying to find the module in each member of sys.path, and file as the result of this lookup (more details in PyMOTW Modules and Imports.). So if the module is located in an absolute path in sys.path the result is absolute, but if it is located in a relative path in sys.path the result is relative.
Now the site.py startup file takes care of delivering only absolute path in sys.path, except the initial '', so if you don’t change it by other means than setting the PYTHONPATH (whose path are also made absolute, before prefixing sys.path), you will get always an absolute path, but when the module is accessed through the current directory.
Now if you trick sys.path in a funny way you can get anything.
As example if you have a sample module foo.py in /tmp/ with the code:
import sys
print(sys.path)
print (__file__)
If you go in /tmp you get:
>>> import foo
['', '/tmp', '/usr/lib/python3.3', ...]
./foo.py
When in in /home/user, if you add /tmp your PYTHONPATH you get:
>>> import foo
['', '/tmp', '/usr/lib/python3.3', ...]
/tmp/foo.py
Even if you add ../../tmp, it will be normalized and the result is the same.
But if instead of using PYTHONPATH you use directly some funny path
you get a result as funny as the cause.
>>> import sys
>>> sys.path.append('../../tmp')
>>> import foo
['', '/usr/lib/python3.3', .... , '../../tmp']
../../tmp/foo.py
Guido explains in the above cited thread, why python do not try to transform all entries in absolute paths:
we don’t want to have to call getpwd() on every import ….
getpwd() is relatively slow and can sometimes fail outright,
So your path is used as it is.
__file__ is absolute since Python 3.4, except when executing a script directly using a relative path:
Module
__file__attributes (and related values) should now always contain absolute paths by default, with the sole exception of__main__.__file__when a script has been executed directly using a relative path. (Contributed by Brett Cannon in bpo-18416.)
Not sure if it resolves symlinks though.
Example of passing a relative path:
$ python script.py
Late simple example:
from os import path, getcwd, chdir
def print_my_path():
print('cwd: {}'.format(getcwd()))
print('__file__:{}'.format(__file__))
print('abspath: {}'.format(path.abspath(__file__)))
print_my_path()
chdir('..')
print_my_path()
Under Python-2.*, the second call incorrectly determines the path.abspath(__file__) based on the current directory:
cwd: C:codespy
__file__:cwd_mayhem.py
abspath: C:codespycwd_mayhem.py
cwd: C:codes
__file__:cwd_mayhem.py
abspath: C:codescwd_mayhem.py
As noted by @techtonik, in Python 3.4+, this will work fine since __file__ returns an absolute path.
__file__ can be relative or absolute depending on the python version used and whether the module is executed directly or not.
TL;DR:
- From python 3.5 to 3.8,
__file__is set to the relative path of the module w.r.t. the current working directory if the module is called directly. Otherwise it is set to the absolute path. - Since python 3.9,
__file__is set to the absolute path even if the corresponding module is executed directly.
For instance, having the following setup (inspired by this answer):
# x.py:
from pathlib import Path
import y
print(__file__)
print(Path(__file__))
print(Path(__file__).resolve())
# y.py:
from pathlib import Path
print(__file__)
print(Path(__file__))
with x.py and y.py in the same directory. The different outputs after going to this directory and executing:
python x.py
are:
Python 3.5 – 3.8:
D:py_testsy.py
D:py_testsy.py
x.py
x.py
D:py_testsx.py
Python 3.9 – 3.11
D:py_testsy.py
D:py_testsy.py
D:py_testsx.py
D:py_testsx.py
D:py_testsx.py