What's the reverse of shlex.split?
Question:
How can I reverse the results of a shlex.split? That is, how can I obtain a quoted string that would “resemble that of a Unix shell”, given a list of strings I wish quoted?
Update0
I’ve located a Python bug, and made corresponding feature requests here.
Answers:
How about using pipes.quote?
import pipes
strings = ["ls", "/etc/services", "file with spaces"]
" ".join(pipes.quote(s) for s in strings)
# "ls /etc/services 'file with spaces'"
.
subprocess uses subprocess.list2cmdline(). It’s not an official public API, but it’s mentioned in the subprocess documentation and I think it’s pretty safe to use. It’s more sophisticated than pipes.open() (for better or worse).
We now (3.3) have a shlex.quote function. It’s none other that pipes.quote moved and documented (code using pipes.quote will still work). See http://bugs.python.org/issue9723 for the whole discussion.
subprocess.list2cmdline is a private function that should not be used. It could however be moved to shlex and made officially public. See also http://bugs.python.org/issue1724822.
There is a feature request for adding shlex.join(), which would do exactly what you ask. As of now, there does not seem any progress on it, though, mostly as it would mostly just forward to shlex.quote(). In the bug report, a suggested implementation is mentioned:
' '.join(shlex.quote(x) for x in split_command)
It’s shlex.join() in python 3.8
While shlex.quote is available in Python 3.3 and shlex.join is available in Python 3.8, they will not always serve as a true "reversal" of shlex.split. Observe the following snippet:
import shlex
command = "cd /home && bash -c 'echo $HOME'"
print(shlex.split(command))
# ['cd', '/home', '&&', 'bash', '-c', 'echo $HOME']
print(shlex.join(shlex.split(command)))
# cd /home '&&' bash -c 'echo $HOME'
Notice that after splitting and then joining, the && token now has single quotes around it. If you tried running the command now, you’d get an error: cd: too many arguments
If you use subprocess.list2cmdline() as others have suggested, it works nicer with bash operators like &&:
import subprocess
print(subprocess.list2cmdline(shlex.split(command)))
# cd /home && bash -c "echo $HOME"
However you may notice now that the quotes are now double instead of single. This results in $HOME being expanded by the shell rather than being printed verbatim as if you had used single quotes.
In conclusion, there is no 100% fool-proof way of undoing shlex.split, and you will have to choose the option that best suites your purpose and watch out for edge cases.
How can I reverse the results of a shlex.split? That is, how can I obtain a quoted string that would “resemble that of a Unix shell”, given a list of strings I wish quoted?
Update0
I’ve located a Python bug, and made corresponding feature requests here.
How about using pipes.quote?
import pipes
strings = ["ls", "/etc/services", "file with spaces"]
" ".join(pipes.quote(s) for s in strings)
# "ls /etc/services 'file with spaces'"
.
subprocess uses subprocess.list2cmdline(). It’s not an official public API, but it’s mentioned in the subprocess documentation and I think it’s pretty safe to use. It’s more sophisticated than pipes.open() (for better or worse).
We now (3.3) have a shlex.quote function. It’s none other that pipes.quote moved and documented (code using pipes.quote will still work). See http://bugs.python.org/issue9723 for the whole discussion.
subprocess.list2cmdline is a private function that should not be used. It could however be moved to shlex and made officially public. See also http://bugs.python.org/issue1724822.
There is a feature request for adding shlex.join(), which would do exactly what you ask. As of now, there does not seem any progress on it, though, mostly as it would mostly just forward to shlex.quote(). In the bug report, a suggested implementation is mentioned:
' '.join(shlex.quote(x) for x in split_command)
It’s shlex.join() in python 3.8
While shlex.quote is available in Python 3.3 and shlex.join is available in Python 3.8, they will not always serve as a true "reversal" of shlex.split. Observe the following snippet:
import shlex
command = "cd /home && bash -c 'echo $HOME'"
print(shlex.split(command))
# ['cd', '/home', '&&', 'bash', '-c', 'echo $HOME']
print(shlex.join(shlex.split(command)))
# cd /home '&&' bash -c 'echo $HOME'
Notice that after splitting and then joining, the && token now has single quotes around it. If you tried running the command now, you’d get an error: cd: too many arguments
If you use subprocess.list2cmdline() as others have suggested, it works nicer with bash operators like &&:
import subprocess
print(subprocess.list2cmdline(shlex.split(command)))
# cd /home && bash -c "echo $HOME"
However you may notice now that the quotes are now double instead of single. This results in $HOME being expanded by the shell rather than being printed verbatim as if you had used single quotes.
In conclusion, there is no 100% fool-proof way of undoing shlex.split, and you will have to choose the option that best suites your purpose and watch out for edge cases.