Why is setuptools so slow


I have a one python library but and it is in wheel 40 MB. But, it takes more then 12 hours to make the wheel file.

My PC Configuration:
I have a decent 3.4 GHZ 16 core processor with 3 cache layers and 64 GB RAM and SSD.

setuptools get stuck here:

copying serverviewsgenericedit.py -> buildlibserverviewsgeneric
copying serverviewsgenericlist.py -> buildlibserverviewsgeneric
copying serverviewsgeneric__init__.py -> buildlibserverviewsgeneric
creating buildlibserverviewstemplates
copying serverviewstemplates__init__.py -> buildlibserverviewstemplates

here it is stuck for most of the time but it still uses my processor.

I was just curious why it takes so long to make the Wheel file to post it on pypi.

Asked By: Matthijs990



python only uses 1 core unless you use threads or the concurrent.futures.ProcessPoolExecutor class so for setuptools it does not matter if you have 1 or 16 cores becuse it will only use 1 core(at least as far as I know setuptools does not use threads or so that split the task over more then one core).


After knowing what version of setuptools you use upgrading setuptools with using this command in the terminal: python -m pip install -U setuptools to upgrade setuptools to the newest version that will make it much faster becuse of a bug
that is fixed now but was around then.

Answered By: Matthijs990

When you go to build a package your could to

clean the cache:

find . | grep -E "(/__pycache__$|.pyc$|.pyo$)" | xargs rm -rf

Clean dist and build:

rm -rf dist/
rm -rf build/

In your setup.py add only the package that you need or exclude that you don’t need:

packages=find_namespace_packages(exclude=("test", "doc", "dist", "build"))
Answered By: Tlaloc-ES
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.