Why is setuptools so slow

Question

I have a one python library but and it is in wheel 40 MB. But, it takes more then 12 hours to make the wheel file.

My PC Configuration:
I have a decent 3.4 GHZ 16 core processor with 3 cache layers and 64 GB RAM and SSD.

setuptools get stuck here:

copying serverviewsgenericedit.py -> buildlibserverviewsgeneric
copying serverviewsgenericlist.py -> buildlibserverviewsgeneric
copying serverviewsgeneric__init__.py -> buildlibserverviewsgeneric
creating buildlibserverviewstemplates
copying serverviewstemplates__init__.py -> buildlibserverviewstemplates

here it is stuck for most of the time but it still uses my processor.

I was just curious why it takes so long to make the Wheel file to post it on pypi.

Asked By: Matthijs990

||

Source

Answer 1

python only uses 1 core unless you use threads or the concurrent.futures.ProcessPoolExecutor class so for setuptools it does not matter if you have 1 or 16 cores becuse it will only use 1 core(at least as far as I know setuptools does not use threads or so that split the task over more then one core).

edit:

After knowing what version of setuptools you use upgrading setuptools with using this command in the terminal: python -m pip install -U setuptools to upgrade setuptools to the newest version that will make it much faster becuse of a bug
that is fixed now but was around then.

Answered By: Matthijs990

Answer 2

When you go to build a package your could to

clean the cache:

find . | grep -E "(/__pycache__$|.pyc$|.pyo$)" | xargs rm -rf

Clean dist and build:

rm -rf dist/
rm -rf build/

In your setup.py add only the package that you need or exclude that you don’t need:

packages=find_namespace_packages(exclude=("test", "doc", "dist", "build"))

Answered By: Tlaloc-ES

Why is setuptools so slow

Question:

Answers: