Why does "python setup.py sdist" create unwanted "PROJECT-egg.info" in project root directory?
Question:
When I run
python setup.py sdist
it creates an sdist in my ./dist directory. This includes a “PROJECT-egg.info” file in the zip inside my “dist” folder, which I don’t use, but it doesn’t hurt me, so I just ignore it.
My question is why does it also create a “PROJECT-egg.info” folder in my project root directory? Can I make it stop creating this? If not, can I just delete it immediately after creating the sdist?
I’m using the ‘setup’ function imported from setuptools.
WindowsXP, Python2.7, Setuptools 0.6c11, Distribute 0.6.14.
My setup config looks like:
{'author': 'Jonathan Hartley',
'author_email': '[email protected]',
'classifiers': ['Development Status :: 1 - Planning',
'Intended Audience :: Developers',
'License :: OSI Approved :: BSD License',
'Operating System :: Microsoft :: Windows',
'Programming Language :: Python :: 2.7'],
'console': [{'script': 'demo.py'}],
'data_files': [('Microsoft.VC90.CRT',
['..\lib\Microsoft.VC90.CRT\Microsoft.VC90.CRT.manifest',
'..\lib\Microsoft.VC90.CRT\msvcr90.dll'])],
'description': 'Utilities for games and OpenGL graphics, built around Pyglet.n',
'keywords': '',
'license': 'BSD',
'long_description': "blah blah blah",
'name': 'pygpen',
'options': {'py2exe': {'ascii': True,
'bundle_files': 1,
'dist_dir': 'dist/pygpen-0.1-windows',
'dll_excludes': [],
'excludes': ['_imaging_gif',
'_scproxy',
'clr',
'dummy.Process',
'email',
'email.base64mime',
'email.utils',
'email.Utils',
'ICCProfile',
'Image',
'IronPythonConsole',
'modes.editingmodes',
'startup',
'System',
'System.Windows.Forms.Clipboard',
'_hashlib',
'_imaging',
'_multiprocessing',
'_ssl',
'_socket',
'bz2',
'pyexpat',
'pyreadline',
'select',
'win32api',
'win32pipe',
'calendar',
'cookielib',
'difflib',
'doctest',
'locale',
'optparse',
'pdb',
'pickle',
'pyglet.window.xlib',
'pyglet.window.carbon',
'pyglet.window.carbon.constants',
'pyglet.window.carbon.types',
'subprocess',
'tarfile',
'threading',
'unittest',
'urllib',
'urllib2',
'win32con',
'zipfile'],
'optimize': 2}},
'packages': ['pygpen'],
'scripts': ['demo.py'],
'url': 'http://code.google.com/p/edpath/',
'version': '0.1',
'zipfile': None}
Answers:
This directory is created intentionally as part of the build process for a source distribution. A little gander at the developer guide for setuptools gives you a hint as to why:
But, be sure to ignore any part of the
distutils documentation that deals
with MANIFEST or how it’s generated
from MANIFEST.in; setuptools shields
you from these issues and doesn’t work
the same way in any case. Unlike the
distutils, setuptools regenerates the
source distribution manifest file
every time you build a source
distribution, and it builds it inside
the project’s .egg-info directory, out
of the way of your main project
directory. You therefore need not
worry about whether it is up-to-date
or not.
You may safely delete the directory after your build has completed.
Bonus edit:
I customize the clean command within my setup.py on many of my Python projects to delete *.egg-info, dist, build, and *.pyc and other files. Here’s an example of how it’s done in setup.py:
import os
from setuptools import setup, Command
class CleanCommand(Command):
"""Custom clean command to tidy up the project root."""
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
os.system('rm -vrf ./build ./dist ./*.pyc ./*.tgz ./*.egg-info')
# Further down when you call setup()
setup(
# ... Other setup options
cmdclass={
'clean': CleanCommand,
}
)
To illustrate, after running python setup.py build on a dummy project called “poop” (Yes, I’m very mature), this happens:
$ python setup.py build
running build
running build_py
creating build
creating build/lib
creating build/lib/poop
copying poop/__init__.py -> build/lib/poop
And now if we run python setup.py clean:
$ python setup.py clean
running clean
removed `./build/lib/poop/__init__.py'
removed directory: `./build/lib/poop'
removed directory: `./build/lib'
removed directory: `./build'
Tada!
Note that you can have the PROJECT.egg-info artifacts disappear completely from your sdist.
The command setup.py egg_info will use the source root as the egg base by default, resulting in the PROJECT.egg-info directory being packaged into the sdist.
You can configure the egg base by passing the option --egg-base.
This will create the PROJECT.egg-info directory somewhere else, leaving it out of your source distribution completely. You might also use a setup.cfg to set that property.
The following command to create a sdist without a PROJECT.egg-info works for me:
python setup.py egg_info --egg-base /tmp sdist
Or in a setup.cfg:
[egg_info]
egg_base = /tmp
The -egg.info folder isn’t always a temporary artifact you can delete.
For example, if you use pip install -e YOURPACKAGE for an “editable” install (works via symlink like python setup.py develop so you don’t have to re-install a package every time you edit it locally), the -egg.info folder is required at runtime when your package is imported in another source. If it doesn’t exist, you will get a DistributionNotFound error.
Pythons packaging and build system is broken imho. So there are many hacks and workarounds for things that one would belive work out of the box.
However, the “cleanest” hack I found for deleting the *.egg-info is using the normal clean --all switch along with the egg_info to place the *.egg-info file in a subfolder that will be cleaned by the clean command. Here an example:
In your setup.cfg use something like this:
[egg_info]
egg_base = ./build/lib
where ./build/lib is a folder that clean --all will delete. Then when building your project with setuptools use the clean command with the –all flag, e.g.
python setup.py bdist_wheel clean --all
if you want to build a source bundle as well just make sure to build bdist_wheel before sdist so the build/lib folder exists, e.g.:
python setup.py bdist_wheel sdist clean --all
An alternative solution to jathanism’s method could be the usage of the egg_info hook. I use it to clean up before each build:
from pathlib import Path
from setuptools import setup
from setuptools.command.egg_info import egg_info
here = Path(__file__).parent.resolve()
class CleanEggInfo(egg_info):
def run(self):
shutil.rmtree(here.joinpath('build'), ignore_errors=True)
shutil.rmtree(here.joinpath('dist'), ignore_errors=True)
for d in here.glob('*.egg-info'):
shutil.rmtree(d, ignore_errors=True)
egg_info.run(self)
setup(
cmdclass={
'egg_info': CleanEggInfo,
}
)
When I run
python setup.py sdist
it creates an sdist in my ./dist directory. This includes a “PROJECT-egg.info” file in the zip inside my “dist” folder, which I don’t use, but it doesn’t hurt me, so I just ignore it.
My question is why does it also create a “PROJECT-egg.info” folder in my project root directory? Can I make it stop creating this? If not, can I just delete it immediately after creating the sdist?
I’m using the ‘setup’ function imported from setuptools.
WindowsXP, Python2.7, Setuptools 0.6c11, Distribute 0.6.14.
My setup config looks like:
{'author': 'Jonathan Hartley',
'author_email': '[email protected]',
'classifiers': ['Development Status :: 1 - Planning',
'Intended Audience :: Developers',
'License :: OSI Approved :: BSD License',
'Operating System :: Microsoft :: Windows',
'Programming Language :: Python :: 2.7'],
'console': [{'script': 'demo.py'}],
'data_files': [('Microsoft.VC90.CRT',
['..\lib\Microsoft.VC90.CRT\Microsoft.VC90.CRT.manifest',
'..\lib\Microsoft.VC90.CRT\msvcr90.dll'])],
'description': 'Utilities for games and OpenGL graphics, built around Pyglet.n',
'keywords': '',
'license': 'BSD',
'long_description': "blah blah blah",
'name': 'pygpen',
'options': {'py2exe': {'ascii': True,
'bundle_files': 1,
'dist_dir': 'dist/pygpen-0.1-windows',
'dll_excludes': [],
'excludes': ['_imaging_gif',
'_scproxy',
'clr',
'dummy.Process',
'email',
'email.base64mime',
'email.utils',
'email.Utils',
'ICCProfile',
'Image',
'IronPythonConsole',
'modes.editingmodes',
'startup',
'System',
'System.Windows.Forms.Clipboard',
'_hashlib',
'_imaging',
'_multiprocessing',
'_ssl',
'_socket',
'bz2',
'pyexpat',
'pyreadline',
'select',
'win32api',
'win32pipe',
'calendar',
'cookielib',
'difflib',
'doctest',
'locale',
'optparse',
'pdb',
'pickle',
'pyglet.window.xlib',
'pyglet.window.carbon',
'pyglet.window.carbon.constants',
'pyglet.window.carbon.types',
'subprocess',
'tarfile',
'threading',
'unittest',
'urllib',
'urllib2',
'win32con',
'zipfile'],
'optimize': 2}},
'packages': ['pygpen'],
'scripts': ['demo.py'],
'url': 'http://code.google.com/p/edpath/',
'version': '0.1',
'zipfile': None}
This directory is created intentionally as part of the build process for a source distribution. A little gander at the developer guide for setuptools gives you a hint as to why:
But, be sure to ignore any part of the
distutils documentation that deals
with MANIFEST or how it’s generated
from MANIFEST.in; setuptools shields
you from these issues and doesn’t work
the same way in any case. Unlike the
distutils, setuptools regenerates the
source distribution manifest file
every time you build a source
distribution, and it builds it inside
the project’s .egg-info directory, out
of the way of your main project
directory. You therefore need not
worry about whether it is up-to-date
or not.
You may safely delete the directory after your build has completed.
Bonus edit:
I customize the clean command within my setup.py on many of my Python projects to delete *.egg-info, dist, build, and *.pyc and other files. Here’s an example of how it’s done in setup.py:
import os
from setuptools import setup, Command
class CleanCommand(Command):
"""Custom clean command to tidy up the project root."""
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
os.system('rm -vrf ./build ./dist ./*.pyc ./*.tgz ./*.egg-info')
# Further down when you call setup()
setup(
# ... Other setup options
cmdclass={
'clean': CleanCommand,
}
)
To illustrate, after running python setup.py build on a dummy project called “poop” (Yes, I’m very mature), this happens:
$ python setup.py build
running build
running build_py
creating build
creating build/lib
creating build/lib/poop
copying poop/__init__.py -> build/lib/poop
And now if we run python setup.py clean:
$ python setup.py clean
running clean
removed `./build/lib/poop/__init__.py'
removed directory: `./build/lib/poop'
removed directory: `./build/lib'
removed directory: `./build'
Tada!
Note that you can have the PROJECT.egg-info artifacts disappear completely from your sdist.
The command setup.py egg_info will use the source root as the egg base by default, resulting in the PROJECT.egg-info directory being packaged into the sdist.
You can configure the egg base by passing the option --egg-base.
This will create the PROJECT.egg-info directory somewhere else, leaving it out of your source distribution completely. You might also use a setup.cfg to set that property.
The following command to create a sdist without a PROJECT.egg-info works for me:
python setup.py egg_info --egg-base /tmp sdist
Or in a setup.cfg:
[egg_info]
egg_base = /tmp
The -egg.info folder isn’t always a temporary artifact you can delete.
For example, if you use pip install -e YOURPACKAGE for an “editable” install (works via symlink like python setup.py develop so you don’t have to re-install a package every time you edit it locally), the -egg.info folder is required at runtime when your package is imported in another source. If it doesn’t exist, you will get a DistributionNotFound error.
Pythons packaging and build system is broken imho. So there are many hacks and workarounds for things that one would belive work out of the box.
However, the “cleanest” hack I found for deleting the *.egg-info is using the normal clean --all switch along with the egg_info to place the *.egg-info file in a subfolder that will be cleaned by the clean command. Here an example:
In your setup.cfg use something like this:
[egg_info]
egg_base = ./build/lib
where ./build/lib is a folder that clean --all will delete. Then when building your project with setuptools use the clean command with the –all flag, e.g.
python setup.py bdist_wheel clean --all
if you want to build a source bundle as well just make sure to build bdist_wheel before sdist so the build/lib folder exists, e.g.:
python setup.py bdist_wheel sdist clean --all
An alternative solution to jathanism’s method could be the usage of the egg_info hook. I use it to clean up before each build:
from pathlib import Path
from setuptools import setup
from setuptools.command.egg_info import egg_info
here = Path(__file__).parent.resolve()
class CleanEggInfo(egg_info):
def run(self):
shutil.rmtree(here.joinpath('build'), ignore_errors=True)
shutil.rmtree(here.joinpath('dist'), ignore_errors=True)
for d in here.glob('*.egg-info'):
shutil.rmtree(d, ignore_errors=True)
egg_info.run(self)
setup(
cmdclass={
'egg_info': CleanEggInfo,
}
)