Flask CLI throws 'OSError: [Errno 8] Exec format error' when run through docker-compose
Question:
I’m running a Flask application with a Custom Script. Or trying to, anyway.
I’m on Windows 10 and the application ought to run in a linux Docker container with the command:
docker-compose up api
Docker-compose is version 1.23.2. In the dockerfile, the api service runs via the command:
command: python manage.py run --host "0.0.0.0" --with-threads
As it tries to start up, I see the exception
OSError: [Errno 8] Exec format error: '/api/manage.py'
I initially thought this would be the Dreaded Windows Line Endings, come for me once more, but running dos2unix on all my source files has not resolved the problem.
How can I avoid this error?
manage.py
import click
from flask.cli import FlaskGroup
from my_app_api import create_app
def create_my_app(info):
return create_app()
@click.group(cls=FlaskGroup, create_app=create_my_app)
def cli():
pass
if __name__ == "__main__":
cli()
Full traceback
api_1 | Traceback (most recent call last):
api_1 | File "manage.py", line 22, in <module>
api_1 | cli()
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 764, in __call__
api_1 | return self.main(*args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 380, in main
api_1 | return AppGroup.main(self, *args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 717, in main
api_1 | rv = self.invoke(ctx)
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
api_1 | return _process_result(sub_ctx.command.invoke(sub_ctx))
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 956, in invoke
api_1 | return ctx.invoke(self.callback, **ctx.params)
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
api_1 | return callback(*args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 64, in new_func
api_1 | return ctx.invoke(f, obj, *args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
api_1 | return callback(*args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 438, in run_command
api_1 | use_debugger=debugger, threaded=with_threads)
api_1 | File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 988, in run_simple
api_1 | run_with_reloader(inner, extra_files, reloader_interval, reloader_type)
api_1 | File "/usr/local/lib/python3.6/site-packages/werkzeug/_reloader.py", line 332, in run_with_reloader
api_1 | sys.exit(reloader.restart_with_reloader())
api_1 | File "/usr/local/lib/python3.6/site-packages/werkzeug/_reloader.py", line 176, in restart_with_reloader
api_1 | exit_code = subprocess.call(args, env=new_environ, close_fds=False)
api_1 | File "/usr/local/lib/python3.6/subprocess.py", line 287, in call
api_1 | with Popen(*popenargs, **kwargs) as p:
api_1 | File "/usr/local/lib/python3.6/subprocess.py", line 729, in __init__
api_1 | restore_signals, start_new_session)
api_1 | File "/usr/local/lib/python3.6/subprocess.py", line 1364, in _execute_child
api_1 | raise child_exception_type(errno_num, err_msg, err_filename)
api_1 | OSError: [Errno 8] Exec format error: '/api/manage.py'
Answers:
Looks like your api/manage.py doesn’t have a shebang ([Wikipedia]: Shebang (Unix)), so the default (current) command processor (a shell – typically bash) is attempting to run it, which (obviously) fails.
To correct the problem, add a shebang (at the beginning of the file, making sure that your editor adds the Nix style line ending (n, 0x0A, LF)):
-
Default Python installation:
#!/usr/bin/env python
-
Variant (specify Python 3 explicitly):
#!/usr/bin/env python3
-
Custom Python installation:
#!/full/path/to/your/custom/python/executable
Note that you also need exec permissions on the file (chmod +x api/manage.py).
Example:
[cfati@cfati-5510-0:/cygdrive/e/Work/Dev/StackOverflow/q055271912]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[064bit prompt]> ls
code00.py code01.py
[064bit prompt]>
[064bit prompt]> cat code00.py
print("This is:", __file__)
[064bit prompt]> python3 -c "import os, subprocess;subprocess.Popen(os.path.join(os.getcwd(), "code00.py")).communicate()"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/cygdrive/e/Work/Dev/StackOverflow/q055271912/code00.py'
[064bit prompt]>
[064bit prompt]> cat code01.py
#!/usr/bin/env python3
print("This is:", __file__)
[064bit prompt]> python3 -c "import os, subprocess;subprocess.Popen(os.path.join(os.getcwd(), "code01.py")).communicate()"
This is: /cygdrive/e/Work/Dev/StackOverflow/q055271912/code01.py
Another way would be to run the interpreter followed by the file name, but I don’t know how to do it from Flask – actually that would require patching Werkzeug (_reloader.py: _get_args_for_reloading), but that would be just a lame workaround (gainarie) – see below.
Update #0
Looking at @AxelGrytt’s answer, it turns out it’s a known issue: [GitHub]: pallets/werkzeug – 0.15.0 causes OSError: [Errno 8] Exec format error: in Docker for Windows (hmm, submitted in the same day as this question (and 2 days after the release) đŸ™‚ ).
So, what I have stated above is correct, but it is worth mentioning that there is another way of fixing it: removing the exec permission for the file:
chmod -x api/manage.py
According to Werkzeug authors, from now on, this is desired behavior (also applies to v0.15.2):
- A file with exec permission set, should also have a shebang
- A file without a shebang, shouldn’t have the exec permission set
This is a new behavior in Werkzeug 0.15. Downgrading to Werkzeug 0.14.1 may work, but 0.14 is no longer supported, so you will be better off correcting the issue with your file as described in the other answers.
The answer of @CristiFati worked for me with 1 additional step:
I also had to correct the EOL from rn to n.
Sorry I don’t have enough points to add a comment and have to open a new answer…
If you disable debug mode (don’t pass debug=True or set FLASK_DEBUG=0), the reloader will not be used and so this issue will not happen. The tradeoff is that you no longer have the reloader.
if __name__ == "__main__":
connexion_app.run(host="0.0.0.0", port=constants.API_PORT, debug=True)
It is preferrable to fix this by ensuring files marked as executable have an interpreter line, like #!/usr/bin/env python3 (from https://stackoverflow.com/a/55272071).
Werkzeug==0.15.4 has this question。
pip install --user Werkzeug==0.16
I’m running a Flask application with a Custom Script. Or trying to, anyway.
I’m on Windows 10 and the application ought to run in a linux Docker container with the command:
docker-compose up api
Docker-compose is version 1.23.2. In the dockerfile, the api service runs via the command:
command: python manage.py run --host "0.0.0.0" --with-threads
As it tries to start up, I see the exception
OSError: [Errno 8] Exec format error: '/api/manage.py'
I initially thought this would be the Dreaded Windows Line Endings, come for me once more, but running dos2unix on all my source files has not resolved the problem.
How can I avoid this error?
manage.py
import click
from flask.cli import FlaskGroup
from my_app_api import create_app
def create_my_app(info):
return create_app()
@click.group(cls=FlaskGroup, create_app=create_my_app)
def cli():
pass
if __name__ == "__main__":
cli()
Full traceback
api_1 | Traceback (most recent call last):
api_1 | File "manage.py", line 22, in <module>
api_1 | cli()
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 764, in __call__
api_1 | return self.main(*args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 380, in main
api_1 | return AppGroup.main(self, *args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 717, in main
api_1 | rv = self.invoke(ctx)
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
api_1 | return _process_result(sub_ctx.command.invoke(sub_ctx))
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 956, in invoke
api_1 | return ctx.invoke(self.callback, **ctx.params)
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
api_1 | return callback(*args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 64, in new_func
api_1 | return ctx.invoke(f, obj, *args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
api_1 | return callback(*args, **kwargs)
api_1 | File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 438, in run_command
api_1 | use_debugger=debugger, threaded=with_threads)
api_1 | File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 988, in run_simple
api_1 | run_with_reloader(inner, extra_files, reloader_interval, reloader_type)
api_1 | File "/usr/local/lib/python3.6/site-packages/werkzeug/_reloader.py", line 332, in run_with_reloader
api_1 | sys.exit(reloader.restart_with_reloader())
api_1 | File "/usr/local/lib/python3.6/site-packages/werkzeug/_reloader.py", line 176, in restart_with_reloader
api_1 | exit_code = subprocess.call(args, env=new_environ, close_fds=False)
api_1 | File "/usr/local/lib/python3.6/subprocess.py", line 287, in call
api_1 | with Popen(*popenargs, **kwargs) as p:
api_1 | File "/usr/local/lib/python3.6/subprocess.py", line 729, in __init__
api_1 | restore_signals, start_new_session)
api_1 | File "/usr/local/lib/python3.6/subprocess.py", line 1364, in _execute_child
api_1 | raise child_exception_type(errno_num, err_msg, err_filename)
api_1 | OSError: [Errno 8] Exec format error: '/api/manage.py'
Looks like your api/manage.py doesn’t have a shebang ([Wikipedia]: Shebang (Unix)), so the default (current) command processor (a shell – typically bash) is attempting to run it, which (obviously) fails.
To correct the problem, add a shebang (at the beginning of the file, making sure that your editor adds the Nix style line ending (n, 0x0A, LF)):
-
Default Python installation:
#!/usr/bin/env python-
Variant (specify Python 3 explicitly):
#!/usr/bin/env python3
-
-
Custom Python installation:
#!/full/path/to/your/custom/python/executable
Note that you also need exec permissions on the file (chmod +x api/manage.py).
Example:
[cfati@cfati-5510-0:/cygdrive/e/Work/Dev/StackOverflow/q055271912]> ~/sopr.sh ### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ### [064bit prompt]> ls code00.py code01.py [064bit prompt]> [064bit prompt]> cat code00.py print("This is:", __file__) [064bit prompt]> python3 -c "import os, subprocess;subprocess.Popen(os.path.join(os.getcwd(), "code00.py")).communicate()" Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python3.6/subprocess.py", line 709, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) OSError: [Errno 8] Exec format error: '/cygdrive/e/Work/Dev/StackOverflow/q055271912/code00.py' [064bit prompt]> [064bit prompt]> cat code01.py #!/usr/bin/env python3 print("This is:", __file__) [064bit prompt]> python3 -c "import os, subprocess;subprocess.Popen(os.path.join(os.getcwd(), "code01.py")).communicate()" This is: /cygdrive/e/Work/Dev/StackOverflow/q055271912/code01.py
Another way would be to run the interpreter followed by the file name, but I don’t know how to do it from Flask – actually that would require patching Werkzeug (_reloader.py: _get_args_for_reloading), but that would be just a lame workaround (gainarie) – see below.
Update #0
Looking at @AxelGrytt’s answer, it turns out it’s a known issue: [GitHub]: pallets/werkzeug – 0.15.0 causes OSError: [Errno 8] Exec format error: in Docker for Windows (hmm, submitted in the same day as this question (and 2 days after the release) đŸ™‚ ).
So, what I have stated above is correct, but it is worth mentioning that there is another way of fixing it: removing the exec permission for the file:
chmod -x api/manage.py
According to Werkzeug authors, from now on, this is desired behavior (also applies to v0.15.2):
- A file with exec permission set, should also have a shebang
- A file without a shebang, shouldn’t have the exec permission set
This is a new behavior in Werkzeug 0.15. Downgrading to Werkzeug 0.14.1 may work, but 0.14 is no longer supported, so you will be better off correcting the issue with your file as described in the other answers.
The answer of @CristiFati worked for me with 1 additional step:
I also had to correct the EOL from rn to n.
Sorry I don’t have enough points to add a comment and have to open a new answer…
If you disable debug mode (don’t pass debug=True or set FLASK_DEBUG=0), the reloader will not be used and so this issue will not happen. The tradeoff is that you no longer have the reloader.
if __name__ == "__main__":
connexion_app.run(host="0.0.0.0", port=constants.API_PORT, debug=True)
It is preferrable to fix this by ensuring files marked as executable have an interpreter line, like #!/usr/bin/env python3 (from https://stackoverflow.com/a/55272071).
Werkzeug==0.15.4 has this question。
pip install --user Werkzeug==0.16