How send parameters through a pipe in a python script using argparse?

Question

I have to create a python script that can be used with Linux pipes

I want to run an script where some parameters can be send with a pipe or in the same line

Some examples of the use of my script with the expected output:

echo "a" > list.txt
echo "b" >> list.txt

./run.py p1 p2   # ['p1', 'p2'] expected output
cat list.txt | ./run.py  # ['a', 'b'] expected output
cat list.txt | ./run.py p1 p2 # ['p1', 'p2', 'a', 'b'] expected output

I tried:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('args', nargs=argparse.REMAINDER)
args = parser.parse_args().args
print args

It works only with the parameters in the same line:

./run.py p1 p2  #['p1', 'p2'] OK
cat list.txt | ./run.py  # []  Not OK
cat list.txt | ./run.py p1 p2 # ['p1', 'p2'] expected output

Asked By: Troncador

||

Source

Answer 1

I find xargs useful in such a case.

I haven’t tried myself, but perhaps

cat list.txt | xargs ./run.py p1 p2

works for you?

In case you need to be specific where the arguments go, you can use the xargs placeholder option -J:

cat list.txt | xargs -J{} ./run.py p1 {} p2

would put “a b” between “p1” and “p2”.

Answered By: user707650

Answer 2

A solution by using only argparse

import argparse
import sys

parser = argparse.ArgumentParser()
parser.add_argument('args', nargs=argparse.REMAINDER)
parser.add_argument('stdin', nargs='?', type=argparse.FileType('r'), default=sys.stdin)
args = parser.parse_args().args

if not sys.stdin.isatty():
    stdin = parser.parse_args().stdin.read().splitlines()
else:
    stdin = []

print(args + stdin)

nargs='?' makes stdin optional and sys.stdin.isatty() checks if sys.stdin is empty

Answered By: etopylight

Answer 3

I recommend not not adding an arg for stdin because it just jacks up your argparse help. Instead, add a regular positional argument, and then if stdin was provided, simply read it and assign to that argument.

#!/usr/bin/env python3

import sys
import argparse


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("-f", "--flag", action="store_true", help="set a flag")
    parser.add_argument("PARAMS", nargs="*")
    args = parser.parse_args()
    if not sys.stdin.isatty():
        #print("stdin detected!")
        args.PARAMS.extend(sys.stdin.read().splitlines())
    print(repr(args))


if __name__ == "__main__":
    main()

This method gives you good help:

$ ./run.py -h          
usage: run.py [-h] [-f] [PARAMS ...]

positional arguments:
  PARAMS

optional arguments:
  -h, --help  show this help message and exit
  -f, --flag  set a flag

And it does what you’re asking for in the question:

./run.py p1 p2                   # Namespace(flag=False, PARAMS=['p1', 'p2'])
printf '%sn' a b | ./run.py -f  # Namespace(flag=True, PARAMS=['a', 'b'])
cat list.txt | ./run.py a b      # Namespace(flag=False, PARAMS=['a', 'b', 'x', 'y', 'z'])

Answered By: mattmc3

Answer 4

So when I first ran across this post I did something similar to mattmc3 answer. However later on I pondered if we couldn’t do this with an custom action, below is the result. It is a lot more code than the other answers but is easier to wrap into a module and reuse in multiple places. Because of the way I use super this only works in Python 3 but could be easily modified to work in Python 2 if you still need it.

#!/usr/bin/env python3

import argparse
import sys

class ExtendFromPipe(argparse._StoreAction):

    def __init__(self, *pargs, **kwargs):
        super().__init__(*pargs, **kwargs)
        # Values from STDIN will extend a list so forcing nargs to '*' will
        # ensure this argument always creates a list.
        self.nargs = '*'

    def __call__(self, parser, namespace, values, option_string=None):
        # Calling super here ensures that there will be a default list
        # After we check to see if the STDIN is coming from a TTY interface
        # if we are being piped information this will be False. We then give
        # a default type conversion if there wasn't one provide and split
        # the input lines from the STDIN and convert them using the type
        # We then get the current value from the name space extend it with
        # the STDIN values and then update the namespace with the new values.
        super().__call__(parser, namespace, values, option_string)
        if not sys.stdin.isatty():
            typecon = self.type if self.type else str
            fromstdin = [typecon(k) for k in sys.stdin.read().splitlines()]
            temp = getattr(namespace, self.dest)
            temp.extend(fromstdin)
            setattr(namespace, self.dest, temp)

if __name__ == "__main__":

    desc = 'Implements Action class that reads from STDIN'
    parser = argparse.ArgumentParser(description=desc)

    parser.add_argument('input', action=ExtendFromPipe)

    cli_args = parser.parse_args()

    print(cli_args.input)

This returns the outputs exactly as requested it will even take care type conversions if passed and all inside the confines of the argparser framework.

Answered By: mechsin

How send parameters through a pipe in a python script using argparse?

Question:

Answers: