Is there a way to compile a python application into static binary?
Question:
What I’m trying to do is ship my code to a remote server, that may have different python version installed and/or may not have packages my app requires.
Right now to achieve such portability I have to build relocatable virtualenv with interpreter and code. That approach has some issues (for example, you have to manually copy a bunch of libraries into your virtualenv, since --always-copy doesn’t work as expected) and generally slow.
There’s (in theory) a way to build python itself statically.
I wonder if I could pack interpreter with my code into one binary and run my application as module. Something like that: ./mypython -m myapp run or ./mypython -m gunicorn -c ./gunicorn.conf myapp.wsgi:application.
Answers:
Freeze options:
However, your target server should have the environment you want -> you should be able to ‘create’ it. If it doesn’t, you should build your software to match the environment.
I found this handy guide on how to install custom version of python to a virtualenv, assuming you have ssh access: https://stackoverflow.com/a/5507373/5616110
In virtualenv, you should be able to pip install anything and you shouldn’t need to worry about sudo privileges. Of course, having those and access to package manager like apt makes everything a lot easier.
You’re probably looking for something like Freeze, which is able to compile your Python application with all its libraries into a static binary:
There are two ways you could go about to solve your problem
- Use a static builder, like freeze, or
pyinstaller, or py2exe
- Compile using
cython
This answer explains how you can go about doing it using the second approach, since the first method is not cross platform and version, and has been explained in other answers. Also, using programs like pyinstaller typically results in huge file sizes, while using cython will result in a file that’s much smaller
-
First, install cython.
sudo -H pip3 install cython
-
Then, you can use cython to generate a C file out of the Python .py file
(in reference to https://stackoverflow.com/a/22040484/5714445)
cython example_file.py --embed
-
Use GCC to compile it after getting your current python version (Note: The below assumes you are trying to compile it to Python3)
PYTHONLIBVER=python$(python3 -c 'import sys; print(".".join(map(str, sys.version_info[:2])))')$(python3-config --abiflags)
gcc -Os $(python3-config --includes) example_file.c -o output_bin_file $(python3-config --ldflags) -l$PYTHONLIBVER
You will now have a binary file output_bin_file, which is what you are looking for
Other things to note:
- Change
example_file.py to whatever file you are actually trying to compile.
- Cython is used to use C-Type Variable definitions for static memory allocation to speed up Python programs. In your case however, you will still be using traditional Python definitions.
- If you are using additional libraries (like
opencv, for example), you might have to provide the directory to them using -L and then specify the name of the library using -l in the GCC Flags. For more information on this, please refer to GCC flags
- The above method might not work for anaconda python, as you will likely have to install a version of gcc that is compatible with your conda-python.
You might wish to investigate Nuitka. It takes python source code and converts it in to C++ API calls. Then it compiles into an executable binary (ELF on Linux). It has been around for a few years now and supports a wide range of Python versions.
You will probably also get a performance improvement if you use it. Recommended.
If you are on a Mac you can use py2app to create a .app bundle, which starts your Django app when you double-click on it.
I described how to bundle Django and CherryPy into such a bundle at https://moosystems.com/articles/14-distribute-django-app-as-native-desktop-app-01.html
In the article I use pywebview to display your Django site in a local application window.
I have created a docker image that relies on Nuitka and a custom statically linked python3.10 to create a static binary.
Did not test it extensively, if you have the chance please let me know if it works for your use case.
You can check it at:
https://github.com/joaompinto/docker-build-python-static-bin
Also you can use the pyinstaller package to create a binary executable file from a Python script.
Here’s an example of how you can use pyinstaller to create a binary executable:
First, install the pyinstaller package using pip:
pip install pyinstaller
Next, navigate to the directory containing your Python script.
Run the following command to create a binary executable:
pyinstaller -F myscript.py
This will create a binary executable file called myscript in the dist directory.
You can also use the –onefile option to create a single file executable, rather than a directory containing multiple files:
pyinstaller --onefile myscript.py
For more advanced usage, you can refer to the pyinstaller documentation: https://pyinstaller.readthedocs.io/en/stable/
What I’m trying to do is ship my code to a remote server, that may have different python version installed and/or may not have packages my app requires.
Right now to achieve such portability I have to build relocatable virtualenv with interpreter and code. That approach has some issues (for example, you have to manually copy a bunch of libraries into your virtualenv, since --always-copy doesn’t work as expected) and generally slow.
There’s (in theory) a way to build python itself statically.
I wonder if I could pack interpreter with my code into one binary and run my application as module. Something like that: ./mypython -m myapp run or ./mypython -m gunicorn -c ./gunicorn.conf myapp.wsgi:application.
Freeze options:
However, your target server should have the environment you want -> you should be able to ‘create’ it. If it doesn’t, you should build your software to match the environment.
I found this handy guide on how to install custom version of python to a virtualenv, assuming you have ssh access: https://stackoverflow.com/a/5507373/5616110
In virtualenv, you should be able to pip install anything and you shouldn’t need to worry about sudo privileges. Of course, having those and access to package manager like apt makes everything a lot easier.
You’re probably looking for something like Freeze, which is able to compile your Python application with all its libraries into a static binary:
There are two ways you could go about to solve your problem
- Use a static builder, like freeze, or
pyinstaller, orpy2exe - Compile using
cython
This answer explains how you can go about doing it using the second approach, since the first method is not cross platform and version, and has been explained in other answers. Also, using programs like pyinstaller typically results in huge file sizes, while using cython will result in a file that’s much smaller
-
First, install
cython.sudo -H pip3 install cython -
Then, you can use
cythonto generate a C file out of the Python.pyfile
(in reference to https://stackoverflow.com/a/22040484/5714445)cython example_file.py --embed -
Use GCC to compile it after getting your current python version (Note: The below assumes you are trying to compile it to Python3)
PYTHONLIBVER=python$(python3 -c 'import sys; print(".".join(map(str, sys.version_info[:2])))')$(python3-config --abiflags) gcc -Os $(python3-config --includes) example_file.c -o output_bin_file $(python3-config --ldflags) -l$PYTHONLIBVER
You will now have a binary file output_bin_file, which is what you are looking for
Other things to note:
- Change
example_file.pyto whatever file you are actually trying to compile. - Cython is used to use C-Type Variable definitions for static memory allocation to speed up Python programs. In your case however, you will still be using traditional Python definitions.
- If you are using additional libraries (like
opencv, for example), you might have to provide the directory to them using-Land then specify the name of the library using-lin the GCC Flags. For more information on this, please refer to GCC flags - The above method might not work for anaconda python, as you will likely have to install a version of gcc that is compatible with your conda-python.
You might wish to investigate Nuitka. It takes python source code and converts it in to C++ API calls. Then it compiles into an executable binary (ELF on Linux). It has been around for a few years now and supports a wide range of Python versions.
You will probably also get a performance improvement if you use it. Recommended.
If you are on a Mac you can use py2app to create a .app bundle, which starts your Django app when you double-click on it.
I described how to bundle Django and CherryPy into such a bundle at https://moosystems.com/articles/14-distribute-django-app-as-native-desktop-app-01.html
In the article I use pywebview to display your Django site in a local application window.
I have created a docker image that relies on Nuitka and a custom statically linked python3.10 to create a static binary.
Did not test it extensively, if you have the chance please let me know if it works for your use case.
You can check it at:
https://github.com/joaompinto/docker-build-python-static-bin
Also you can use the pyinstaller package to create a binary executable file from a Python script.
Here’s an example of how you can use pyinstaller to create a binary executable:
First, install the pyinstaller package using pip:
pip install pyinstaller
Next, navigate to the directory containing your Python script.
Run the following command to create a binary executable:
pyinstaller -F myscript.py
This will create a binary executable file called myscript in the dist directory.
You can also use the –onefile option to create a single file executable, rather than a directory containing multiple files:
pyinstaller --onefile myscript.py
For more advanced usage, you can refer to the pyinstaller documentation: https://pyinstaller.readthedocs.io/en/stable/