Passing a List to Python From Command Line

Question

I would like to make my python script run from the command line when supplies with some arguments. However, one of the arguments should be a list of options specific to one segment of the script. Would string parsing be the only way to do this by actually constructing the list after the “command line list” string is split from commas? If so, how would you go about that?

Example: -details=[‘name’, ‘title’, ‘address’]

Asked By: Sam

||

Source

Answer 1

argparse is nice for this, it’s in the standard library as of 2.7 and 3.2 but otherwise a pip install away.

Your main concern of specifying a variable-length list can be addressed by making the list interpreted as a single argument in the shell by using quotes (could depend on your shell I suppose):

% python prog.py 'name title address' spam

where prog.py contains

import sys
my_list = sys.argv[1].split() 
# my_list is ['name', 'title', 'address']
if 'name' in my_list:
   do_something()

or similar. Use an argument with split to delimit your list:

% python prog.py "you're a foo, lift the bar"

my_list = [x.strip() for x in  sys.argv[1].split(',')]
# my_list is ["you're a foo", "lift the bar"]

But please use argparse instead; especially if you want to use use -c style flags.

One way to interpret your question is:

“I’m already using argparse, since that’s the sensible way to interpret command line arguments in Python. How do I specify that some options are within a specific category?”

In your question you’ve shown an example of something the shells I use of would choke on;

% python prog.py -v -details=['name', 'title', 'address'] --quickly -t 4

wouldn’t make it to python to be parsed because they’d use spaces to separate arguments and might use [ and ] as shell syntax.

I suggest the following instead

% python prog.py -v --details name title address --quickly -t 4

where a prog.py file of

import argparse

parser = argparse.ArgumentParser() 
parser.add_argument('-v', action='store_true')
parser.add_argument('--details', nargs='*')
parser.add_argument('--quickly', action='store_true')
parser.add_argument('-t')

args = parser.parse_args()
#args is Namespace(details=['asdf', 'a', 'a'], quickly=False, t='4', v=True)
details = args.details
#details is ['asdf', 'a', 'a']

Now, as per your question, you didn’t have to do the string parsing yourself.

Answered By: Thomas

Answer 2

Yes, argparse is your best bet, and if you want to provide a list of values to one of your named arguments, it looks like this (the nargs parameter is the key to this):

>>> import argparse
>>> arg_parser = argparse.ArgumentParser()
>>> arg_parser.add_argument('--details',
                            nargs='*',
                            type=str,
                            default=[],
                            help='a list of the details')

# your args on the command line like this example
>>> the_args = arg_parser.parse_args("--details 'name' 'title' 'address'".split())
>>> print the_args.details
["'name'", "'title'", "'address'"])

Answered By: Mark Gemmill

Answer 3

Program:

import sys, ast, getopt, types

def main(argv):            
    arg_dict={}
    switches={'li':list,'di':dict,'tu':tuple}
    singles=''.join([x[0]+':' for x in switches])
    long_form=[x+'=' for x in switches]
    d={x[0]+':':'--'+x for x in switches}
    try:            
        opts, args = getopt.getopt(argv, singles, long_form)
    except getopt.GetoptError:          
        print "bad arg"                       
        sys.exit(2)       

    for opt, arg in opts:        
        if opt[1]+':' in d: o=d[opt[1]+':'][2:]
        elif opt in d.values(): o=opt[2:]
        else: o =''
        print opt, arg,o
        if o and arg:
            arg_dict[o]=ast.literal_eval(arg)

        if not o or not isinstance(arg_dict[o], switches[o]):    
            print opt, arg, " Error: bad arg"
            sys.exit(2)                 

    for e in arg_dict:
        print e, arg_dict[e], type(arg_dict[e])        

if __name__ == '__main__':
    main(sys.argv[1:])

Command line:

python py.py --l='[1,2,3,[1,2,3]]' -d "{1:'one',2:'two',3:'three'}" --tu='(1,2,3)'

Output:

args:  ['--l=[1,2,3,[1,2,3]]', '-d', "{1:'one',2:'two',3:'three'}", '--tu=(1,2,3)']
tu (1, 2, 3) <type 'tuple'>
di {1: 'one', 2: 'two', 3: 'three'} <type 'dict'>
li [1, 2, 3, [1, 2, 3]] <type 'list'>

This code snippet will take short or long command switches like -l or --li= and parse the text after the switch into a Python data structure like a list, tuple or a dict. The parsed data structure ends up in a dictionary with the long-form switch key.

Using ast.literal_eval is relatively safe. It can only parse python data definitions.

Answered By: the wolf

Answer 4

I really like the-wolf‘s approach using variable length collections as explicit string arguments.

In my opinion, nargs='*' has noteworthy drawbacks: when attempting to collect strings as positional arguments (at least one string must be present), or if you are trying to use subparsers, you will find that nargs='*' and nargs='+' use greedy completion, and they don’t seem to stop consuming for any good reasons. Even if syntax for an optional argument or a number comes up, the string() type will keep consuming. (This gets harder to anticipate with subparsers).

Best case, arguments placed after (positional and optional) are being ignored, and worse, you are likely passing corrupt data types to the argparse array.

We should be able to define a custom ActionType that is looking for a quoted string. If it finds one, then we adapt the-wolf‘s examples (nearly verbatim it seems).

This keep things clean in argparse and makes general use of variable collections much less bitchy.

Answered By: user2097818

Passing a List to Python From Command Line

Question:

Answers: