Using module's own objects in __main__.py
Question:
I’m trying to access a module’s data from inside its __main__.py.
The structure is as follows:
mymod/
__init__.py
__main__.py
Now, if I expose a variable in __init__.py like this:
__all__ = ['foo']
foo = {'bar': 'baz'}
How can I access foo from __main__.py?
Answers:
The __init__ module of a package acts like members of the package itself, so the objects are imported directly from mymod:
from mymod import foo
Or
from . import foo
if you like to be terse, then read about relative imports. You need to make sure, as always, that you do not invoke the module as mymod/__main__.py, for example, as that will prevent Python from detecting mymod as a package. You may wish to look into distutils.
You need to either have the package already in sys.path, add the directory containing mymod to sys.path in __main__.py, or use the -m switch.
To add mymod to the path would look something like this (in __main__.py):
import sys
import os
path = os.path.dirname(sys.modules[__name__].__file__)
path = os.path.join(path, '..')
sys.path.insert(0, path)
from myprog import function_you_referenced_from_init_file
Using the -m switch would like:
python -m mymod
See this answer for more discussion.
Module directory structure is as follows:
py/
__init__.py
__main__.py
__init__.py
#!/usr/bin/python3
#
# __init__.py
#
__all__ = ['foo']
foo = {'bar': 'baz'}
info = { "package": __package__,
"name": __name__,
"locals": [x for x in locals().copy()] }
print(info)
__main__.py
#!/usr/bin/python3
#
# __main__.py
#
info = { "package": __package__,
"name": __name__,
"locals": [x for x in locals().copy()] }
print(info)
from . import info as pyinfo
print({"pyinfo: ": pyinfo})
Execute the module as a script using the -m flag
$ python -m py
# the printout from the 'print(info)' command in __init__.py
{'name': 'py', 'locals': ['__all__', '__builtins__', '__file__', '__package__', '__path__', '__name__', 'foo', '__doc__'], 'package': None}
# the printout from the 'print(info)' command in __main__.py
{'name': '__main__', 'locals': ['__builtins__', '__name__', '__file__', '__loader__', '__doc__', '__package__'], 'package': 'py'}
# the printout from the 'print(pyinfo)' command in __main__.py
{'pyinfo: ': {'name': 'py', 'locals': ['__all__', '__builtins__', '__file__', '__package__', '__path__', '__name__', 'foo', '__doc__'], 'package': None}}
If you run the module with python -m mymod then code in __main__.py will be able to import from the rest of the module without having to add the module to sys.path.
The issue I run into the most with this type of thing is that I often want to run the __init__.py file as a script to test features, but these should not be run when loading the package. There is a useful workaround for the different execution paths between python <package>/__init__.py and python -m <package>.
$ python -m <module> executes <package>/__main__.py. __init__.py is not loaded.$ python <package>/__init__.py simply executes the script __init__.py like a normal script.
The problem
When we want __init__.py to have an if __name__ == '__main__': ... clause that uses stuff from __main__.py. We can’t import __main__.py because it will always import __main__.pyc from the interpreter’s path. (Unless…we resort to absolute path import hacks, which can cause a lot of other mess).
The solution A solution 🙂
Use two script files for the module’s __main__:
<package>/
__init__.py
__main__.py
main.py
# __init__.py
# ...
# some code, including module methods and __all__ definitions
__all__ = ['foo', 'bar']
bar = {'key': 'value'}
def foo():
return bar
# ...
if __name__ == '__main__':
from main import main
main.main()
# __main__.py
# some code...such as:
import sys
if (len(sys.argv) > 1 and sys.argv[1].lower() == 'option1'):
from main import main()
main('option1')
elif (len(sys.argv) > 1 and sys.argv[1].lower() == 'option2'):
from main import main()
main('option2')
else:
# do something else?
print 'invalid option. please use "python -m <package> option1|option2"'
# main.py
def main(opt = None):
if opt == 'option1':
from __init__ import foo
print foo()
elif opt == 'option2':
from __init__ import bar
print bar.keys()
elif opt is None:
print 'called from __init__'
The imports in main.py are probably not ideal in the case we are running from __init__.py, as we are reloading them into the local scope of another module, despite having loading them in __init__.py already, but the explicit loading should avoid circular loading. If you do load the entire __init__ module again in your main.py, it will not be loaded as __main__, so should be safe as far as circular loading is concerned.
I found the first answer to be useful (i.e., hacking sys.path), but with the addition of pathlib in Python 3.4, I found the following code to be much more simple and Pythonic:
import sys
from pathlib import Path
# You don't need to .insert(), just append
sys.path.append(str(Path(__file__).parent.parent))
I’m trying to access a module’s data from inside its __main__.py.
The structure is as follows:
mymod/
__init__.py
__main__.py
Now, if I expose a variable in __init__.py like this:
__all__ = ['foo']
foo = {'bar': 'baz'}
How can I access foo from __main__.py?
The __init__ module of a package acts like members of the package itself, so the objects are imported directly from mymod:
from mymod import foo
Or
from . import foo
if you like to be terse, then read about relative imports. You need to make sure, as always, that you do not invoke the module as mymod/__main__.py, for example, as that will prevent Python from detecting mymod as a package. You may wish to look into distutils.
You need to either have the package already in sys.path, add the directory containing mymod to sys.path in __main__.py, or use the -m switch.
To add mymod to the path would look something like this (in __main__.py):
import sys
import os
path = os.path.dirname(sys.modules[__name__].__file__)
path = os.path.join(path, '..')
sys.path.insert(0, path)
from myprog import function_you_referenced_from_init_file
Using the -m switch would like:
python -m mymod
See this answer for more discussion.
Module directory structure is as follows:
py/
__init__.py
__main__.py
__init__.py
#!/usr/bin/python3
#
# __init__.py
#
__all__ = ['foo']
foo = {'bar': 'baz'}
info = { "package": __package__,
"name": __name__,
"locals": [x for x in locals().copy()] }
print(info)
__main__.py
#!/usr/bin/python3
#
# __main__.py
#
info = { "package": __package__,
"name": __name__,
"locals": [x for x in locals().copy()] }
print(info)
from . import info as pyinfo
print({"pyinfo: ": pyinfo})
Execute the module as a script using the -m flag
$ python -m py
# the printout from the 'print(info)' command in __init__.py
{'name': 'py', 'locals': ['__all__', '__builtins__', '__file__', '__package__', '__path__', '__name__', 'foo', '__doc__'], 'package': None}
# the printout from the 'print(info)' command in __main__.py
{'name': '__main__', 'locals': ['__builtins__', '__name__', '__file__', '__loader__', '__doc__', '__package__'], 'package': 'py'}
# the printout from the 'print(pyinfo)' command in __main__.py
{'pyinfo: ': {'name': 'py', 'locals': ['__all__', '__builtins__', '__file__', '__package__', '__path__', '__name__', 'foo', '__doc__'], 'package': None}}
If you run the module with python -m mymod then code in __main__.py will be able to import from the rest of the module without having to add the module to sys.path.
The issue I run into the most with this type of thing is that I often want to run the __init__.py file as a script to test features, but these should not be run when loading the package. There is a useful workaround for the different execution paths between python <package>/__init__.py and python -m <package>.
$ python -m <module>executes<package>/__main__.py.__init__.pyis not loaded.$ python <package>/__init__.pysimply executes the script__init__.pylike a normal script.
The problem
When we want __init__.py to have an if __name__ == '__main__': ... clause that uses stuff from __main__.py. We can’t import __main__.py because it will always import __main__.pyc from the interpreter’s path. (Unless…we resort to absolute path import hacks, which can cause a lot of other mess).
The solution A solution 🙂
Use two script files for the module’s __main__:
<package>/
__init__.py
__main__.py
main.py
# __init__.py
# ...
# some code, including module methods and __all__ definitions
__all__ = ['foo', 'bar']
bar = {'key': 'value'}
def foo():
return bar
# ...
if __name__ == '__main__':
from main import main
main.main()
# __main__.py
# some code...such as:
import sys
if (len(sys.argv) > 1 and sys.argv[1].lower() == 'option1'):
from main import main()
main('option1')
elif (len(sys.argv) > 1 and sys.argv[1].lower() == 'option2'):
from main import main()
main('option2')
else:
# do something else?
print 'invalid option. please use "python -m <package> option1|option2"'
# main.py
def main(opt = None):
if opt == 'option1':
from __init__ import foo
print foo()
elif opt == 'option2':
from __init__ import bar
print bar.keys()
elif opt is None:
print 'called from __init__'
The imports in main.py are probably not ideal in the case we are running from __init__.py, as we are reloading them into the local scope of another module, despite having loading them in __init__.py already, but the explicit loading should avoid circular loading. If you do load the entire __init__ module again in your main.py, it will not be loaded as __main__, so should be safe as far as circular loading is concerned.
I found the first answer to be useful (i.e., hacking sys.path), but with the addition of pathlib in Python 3.4, I found the following code to be much more simple and Pythonic:
import sys
from pathlib import Path
# You don't need to .insert(), just append
sys.path.append(str(Path(__file__).parent.parent))