Cannot pass an argument to python with "#!/usr/bin/env python"
Question:
I needed to have a directly executable python script, so i started the file with #!/usr/bin/env python. However, I also need unbuffered output, so i tried #!/usr/bin/env python -u, but that fails with python -u: no such file or directory.
I found out that #/usr/bin/python -u works, but I need it to get the python in PATH to support virtual env environments.
What are my options?
Answers:
Passing arguments to the shebang line is not standard and in as you have experimented do not work in combination with env in Linux. The solution with bash is to use the builtin command “set” to set the required options. I think you can do the same to set unbuffered output of stdin with a python command.
my2c
This is a kludge and requires bash, but it works:
#!/bin/bash
python -u <(cat <<"EOF"
# Your script here
print "Hello world"
EOF
)
When you use shebang on Linux, the entire rest of the line after the interpreter name is interpreted as a single argument. The python -u gets passed to env as if you’d typed: /usr/bin/env 'python -u'. The /usr/bin/env searches for a binary called python -u, which there isn’t one.
In some environment, env doesn’t split arguments.
So your env is looking for python -u in your path.
We can use sh to work around.
Replace your shebang with the following code lines and everything will be fine.
#!/bin/sh
''''exec python -u -- "$0" ${1+"$@"} # '''
# vi: syntax=python
p.s. we need not worry about the path to sh, right?
It is better to use environment variable to enable this. See python doc : http://docs.python.org/2/using/cmdline.html
for your case:
export PYTHONUNBUFFERED=1
script.py
This might be a little bit outdated but env(1) manual tells one can use ‘-S’ for that case
#!/usr/bin/env -S python -u
It seems to work pretty good on FreeBSD.
Here is a script alternative to /usr/bin/env, that permits passing of arguments on the hash-bang line, based on /bin/bash and with the restriction that spaces are disallowed in the executable path. I call it “envns” (env No Spaces):
#!/bin/bash
ARGS=( $1 ) # separate $1 into multiple space-delimited arguments.
shift # consume $1
PROG=`which ${ARGS[0]}`
unset ARGS[0] # discard executable name
ARGS+=( "$@" ) # remainder of arguments preserved "as-is".
exec $PROG "${ARGS[@]}"
Assuming this script is located at /usr/local/bin/envns, here’s your shebang line:
#!/usr/local/bin/envns python -u
Tested on Ubuntu 13.10 and cygwin x64.
Building off of Larry Cai’s answer, env allows you to set a variable directly in the command line. That means that -u can be replaced by the equivalent PYTHONUNBUFFERED setting before python:
#!/usr/bin/env PYTHONUNBUFFERED="YESSSSS" python
Works on RHEL 6.5. I am pretty sure that feature of env is just about universal.
I recently wrote a patch for the GNU Coreutils version of env to address this issue:
http://lists.gnu.org/archive/html/coreutils/2017-05/msg00018.html
If you have this, you can do:
#!/usr/bin/env :lang:--foo:bar
env will split :lang:foo:--bar into the fields lang, foo and --bar. It will search PATH for the interpreter lang, and then invoke it with arguments --foo, bar, plus the path to the script and that script’s arguments.
There is also a feature to pass the name of the script in the middle of the options. Suppose you want to run lang -f <thecriptname> other-arg, followed by the remaining arguments. With this patched env, it is done like this:
#!/usr/bin/env :lang:-f:{}:other-arg
The leftmost field which is equivalent to {} is replaced with the first argument that follows, which, under hash bang invocation, is the script name. That argument is then removed.
Here, other-arg could be something processed by lang or perhaps something processed by the script.
To understand better, see the numerous echo test cases in the patch.
I chose the : character because it is an existing separator used in PATH on POSIX systems. Since env does PATH searching, it’s vanishingly unlikely to be used for a program whose name contains a colon. The {} marker comes from the find utility, which uses it to denote the insertion of a path into the -exec command line.
I needed to have a directly executable python script, so i started the file with #!/usr/bin/env python. However, I also need unbuffered output, so i tried #!/usr/bin/env python -u, but that fails with python -u: no such file or directory.
I found out that #/usr/bin/python -u works, but I need it to get the python in PATH to support virtual env environments.
What are my options?
Passing arguments to the shebang line is not standard and in as you have experimented do not work in combination with env in Linux. The solution with bash is to use the builtin command “set” to set the required options. I think you can do the same to set unbuffered output of stdin with a python command.
my2c
This is a kludge and requires bash, but it works:
#!/bin/bash
python -u <(cat <<"EOF"
# Your script here
print "Hello world"
EOF
)
When you use shebang on Linux, the entire rest of the line after the interpreter name is interpreted as a single argument. The python -u gets passed to env as if you’d typed: /usr/bin/env 'python -u'. The /usr/bin/env searches for a binary called python -u, which there isn’t one.
In some environment, env doesn’t split arguments.
So your env is looking for python -u in your path.
We can use sh to work around.
Replace your shebang with the following code lines and everything will be fine.
#!/bin/sh
''''exec python -u -- "$0" ${1+"$@"} # '''
# vi: syntax=python
p.s. we need not worry about the path to sh, right?
It is better to use environment variable to enable this. See python doc : http://docs.python.org/2/using/cmdline.html
for your case:
export PYTHONUNBUFFERED=1
script.py
This might be a little bit outdated but env(1) manual tells one can use ‘-S’ for that case
#!/usr/bin/env -S python -u
It seems to work pretty good on FreeBSD.
Here is a script alternative to /usr/bin/env, that permits passing of arguments on the hash-bang line, based on /bin/bash and with the restriction that spaces are disallowed in the executable path. I call it “envns” (env No Spaces):
#!/bin/bash
ARGS=( $1 ) # separate $1 into multiple space-delimited arguments.
shift # consume $1
PROG=`which ${ARGS[0]}`
unset ARGS[0] # discard executable name
ARGS+=( "$@" ) # remainder of arguments preserved "as-is".
exec $PROG "${ARGS[@]}"
Assuming this script is located at /usr/local/bin/envns, here’s your shebang line:
#!/usr/local/bin/envns python -u
Tested on Ubuntu 13.10 and cygwin x64.
Building off of Larry Cai’s answer, env allows you to set a variable directly in the command line. That means that -u can be replaced by the equivalent PYTHONUNBUFFERED setting before python:
#!/usr/bin/env PYTHONUNBUFFERED="YESSSSS" python
Works on RHEL 6.5. I am pretty sure that feature of env is just about universal.
I recently wrote a patch for the GNU Coreutils version of env to address this issue:
http://lists.gnu.org/archive/html/coreutils/2017-05/msg00018.html
If you have this, you can do:
#!/usr/bin/env :lang:--foo:bar
env will split :lang:foo:--bar into the fields lang, foo and --bar. It will search PATH for the interpreter lang, and then invoke it with arguments --foo, bar, plus the path to the script and that script’s arguments.
There is also a feature to pass the name of the script in the middle of the options. Suppose you want to run lang -f <thecriptname> other-arg, followed by the remaining arguments. With this patched env, it is done like this:
#!/usr/bin/env :lang:-f:{}:other-arg
The leftmost field which is equivalent to {} is replaced with the first argument that follows, which, under hash bang invocation, is the script name. That argument is then removed.
Here, other-arg could be something processed by lang or perhaps something processed by the script.
To understand better, see the numerous echo test cases in the patch.
I chose the : character because it is an existing separator used in PATH on POSIX systems. Since env does PATH searching, it’s vanishingly unlikely to be used for a program whose name contains a colon. The {} marker comes from the find utility, which uses it to denote the insertion of a path into the -exec command line.