nargs=* equivalent for options in Click
Question:
Is there an equivalent to argparse‘s nargs='*' functionality for optional arguments in Click?
I am writing a command line script, and one of the options needs to be able to take an unlimited number of arguments, like:
foo --users alice bob charlie --bar baz
So users would be ['alice', 'bob', 'charlie'] and bar would be 'baz'.
In argparse, I can specify multiple optional arguments to collect all of the arguments that follow them by setting nargs='*'.
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--users', nargs='*')
>>> parser.add_argument('--bar')
>>> parser.parse_args('--users alice bob charlie --bar baz'.split())
Namespace(bar='baz', users=['alice', 'bob', 'charlie'])
I know Click allows you to specify an argument to accept unlimited inputs by setting nargs=-1, but when I try to set an optional argument’s nargs to -1, I get:
TypeError: Options cannot have nargs < 0
Is there a way to make Click accept an unspecified number of arguments for an option?
Update:
I need to be able to specify options after the option that takes unlimited arguments.
Update:
@Stephen Rauch’s answer answers this question. However, I don’t recommend using the approach I ask for here. My feature request is intentionally not implemented in Click, since it can result in unexpected behaviors. Click’s recommended approach is to use multiple=True:
@click.option('-u', '--user', 'users', multiple=True)
And in the command line, it will look like:
foo -u alice -u bob -u charlie --bar baz
Answers:
One way to approach what you are after is to inherit from click.Option, and customize the parser.
Custom Class:
import click
class OptionEatAll(click.Option):
def __init__(self, *args, **kwargs):
self.save_other_options = kwargs.pop('save_other_options', True)
nargs = kwargs.pop('nargs', -1)
assert nargs == -1, 'nargs, if set, must be -1 not {}'.format(nargs)
super(OptionEatAll, self).__init__(*args, **kwargs)
self._previous_parser_process = None
self._eat_all_parser = None
def add_to_parser(self, parser, ctx):
def parser_process(value, state):
# method to hook to the parser.process
done = False
value = [value]
if self.save_other_options:
# grab everything up to the next option
while state.rargs and not done:
for prefix in self._eat_all_parser.prefixes:
if state.rargs[0].startswith(prefix):
done = True
if not done:
value.append(state.rargs.pop(0))
else:
# grab everything remaining
value += state.rargs
state.rargs[:] = []
value = tuple(value)
# call the actual process
self._previous_parser_process(value, state)
retval = super(OptionEatAll, self).add_to_parser(parser, ctx)
for name in self.opts:
our_parser = parser._long_opt.get(name) or parser._short_opt.get(name)
if our_parser:
self._eat_all_parser = our_parser
self._previous_parser_process = our_parser.process
our_parser.process = parser_process
break
return retval
Using Custom Class:
To use the custom class, pass the cls parameter to @click.option() decorator like:
@click.option("--an_option", cls=OptionEatAll)
or if it is desired that the option will eat the entire rest of the command line, not respecting other options:
@click.option("--an_option", cls=OptionEatAll, save_other_options=False)
How does this work?
This works because click is a well designed OO framework. The @click.option() decorator usually instantiates a
click.Option object but allows this behavior to be over ridden with the cls parameter. So it is a relatively
easy matter to inherit from click.Option in our own class and over ride the desired methods.
In this case we over ride click.Option.add_to_parser() and the monkey patch the parser so that we can
eat more than one token if desired.
Test Code:
@click.command()
@click.option('-g', 'greedy', cls=OptionEatAll, save_other_options=False)
@click.option('--polite', cls=OptionEatAll)
@click.option('--other')
def foo(polite, greedy, other):
click.echo('greedy: {}'.format(greedy))
click.echo('polite: {}'.format(polite))
click.echo('other: {}'.format(other))
if __name__ == "__main__":
commands = (
'-g a b --polite x',
'-g a --polite x y --other o',
'--polite x y --other o',
'--polite x -g a b c --other o',
'--polite x --other o -g a b c',
'-g a b c',
'-g a',
'-g',
'extra',
'--help',
)
import sys, time
time.sleep(1)
print('Click Version: {}'.format(click.__version__))
print('Python Version: {}'.format(sys.version))
for cmd in commands:
try:
time.sleep(0.1)
print('-----------')
print('> ' + cmd)
time.sleep(0.1)
foo(cmd.split())
except BaseException as exc:
if str(exc) != '0' and
not isinstance(exc, (click.ClickException, SystemExit)):
raise
Test Results:
Click Version: 6.7
Python Version: 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
-----------
> -g a b --polite x
greedy: ('a', 'b', '--polite', 'x')
polite: None
other: None
-----------
> -g a --polite x y --other o
greedy: ('a', '--polite', 'x', 'y', '--other', 'o')
polite: None
other: None
-----------
> --polite x y --other o
greedy: None
polite: ('x', 'y')
other: o
-----------
> --polite x -g a b c --other o
greedy: ('a', 'b', 'c', '--other', 'o')
polite: ('x',)
other: None
-----------
> --polite x --other o -g a b c
greedy: ('a', 'b', 'c')
polite: ('x',)
other: o
-----------
> -g a b c
greedy: ('a', 'b', 'c')
polite: None
other: None
-----------
> -g a
greedy: ('a',)
polite: None
other: None
-----------
> -g
Error: -g option requires an argument
-----------
> extra
Usage: test.py [OPTIONS]
Error: Got unexpected extra argument (extra)
-----------
> --help
Usage: test.py [OPTIONS]
Options:
-g TEXT
--polite TEXT
--other TEXT
--help Show this message and exit.
You can use this trick.
import click
@click.command()
@click.option('--users', nargs=0, required=True)
@click.argument('users', nargs=-1)
@click.option('--bar')
def fancy_command(users, bar):
users_str = ', '.join(users)
print('Users: {}. Bar: {}'.format(users_str, bar))
if __name__ == '__main__':
fancy_command()
Add fake option with a needed name and none arguments nargs=0, then add ‘argument’ with the unlimited args nargs=-1.
$ python foo --users alice bob charlie --bar baz
Users: alice, bob, charlie. Bar: baz
But be careful with the further options:
$ python foo --users alice bob charlie --bar baz faz
Users: alice, bob, charlie, faz. Bar: baz
I ran into the same issue. Instead of implementing a single command line option with n number of arguments, I decided to use multiple of the same command line option and just letting Click make a tuple out of the arguments under the hood. I ultimately figured if Click didn’t support it, that decision was probably made for a good reason.
https://click.palletsprojects.com/en/7.x/options/#multiple-options
here is an example of what I am saying:
instead of passing a single string argument a splitting on a delimiter:
commit -m foo:bar:baz
I opted to use this:
commit -m foo -m bar -m baz
here is the source code:
@click.command()
@click.option('--message', '-m', multiple=True)
def commit(message):
click.echo('n'.join(message))
This is more to type, but I do think it makes the CLI more user friendly and robust.
I needed this for myself and thought of settling for the solution provided by @nikita-malovichko, though it is very restrictive, but it didn’t work for me (see my comment to that answer) so came up with the below alternative.
My solution doesn’t directly address the question on how to support nargs=*, but it provided a good alternative for myself so sharing it for the benefit of others.
The idea is to use one option that specifies the expected count for another, i.e., set the nargs count dynamically at runtime. Here is a quick demo:
import click
def with_dynamic_narg(cnt_opt, tgt_opt):
class DynamicNArgSetter(click.Command):
def parse_args(self, ctx, args):
ctx.resilient_parsing = True
parser = self.make_parser(ctx)
opts, _, _ = parser.parse_args(args=list(args))
for p in self.params:
if isinstance(p, click.Option) and p.name == tgt_opt:
p.nargs = int(opts[cnt_opt])
ctx.resilient_parsing = False
return super().parse_args(ctx, args)
return DynamicNArgSetter
@click.command(cls=with_dynamic_narg('c', 'n'))
@click.option("-c", type=click.INT)
@click.option("-n", nargs=0)
def f(c, n):
print(c, n)
if __name__ == '__main__':
f()
In the above code, a custom Command class is created that knows the link between the "count" arg and the target arg that takes multiple args. It first does a local parsing in "resilient" mode to detect the count, then uses the count to update the nargs value of the target arg and then resumes parsing in normal mode.
Here is some sample interaction:
$ python t.py -c 0
0 None
$ python t.py -c 1
Usage: t.py [OPTIONS]
Try 't.py --help' for help.
Error: Missing option '-n'.
$ python t.py -c 0 -n a
Usage: t.py [OPTIONS]
Try 't.py --help' for help.
Error: Got unexpected extra argument (a)
$ python t.py -c 1 -n a
1 a
$ python /tmp/t.py -c 2 -n a b
2 ('a', 'b')
Note: The advantage over the official recommendation of using multiple=True is that we can use filename wildcards and let shell expand them. E.g.,
$ touch abc.1 abc.2
$ python t.py -c $(echo abc.* | wc -w) -n abc.*
2 ('abc.1', 'abc.2')
Is there an equivalent to argparse‘s nargs='*' functionality for optional arguments in Click?
I am writing a command line script, and one of the options needs to be able to take an unlimited number of arguments, like:
foo --users alice bob charlie --bar baz
So users would be ['alice', 'bob', 'charlie'] and bar would be 'baz'.
In argparse, I can specify multiple optional arguments to collect all of the arguments that follow them by setting nargs='*'.
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--users', nargs='*')
>>> parser.add_argument('--bar')
>>> parser.parse_args('--users alice bob charlie --bar baz'.split())
Namespace(bar='baz', users=['alice', 'bob', 'charlie'])
I know Click allows you to specify an argument to accept unlimited inputs by setting nargs=-1, but when I try to set an optional argument’s nargs to -1, I get:
TypeError: Options cannot have nargs < 0
Is there a way to make Click accept an unspecified number of arguments for an option?
Update:
I need to be able to specify options after the option that takes unlimited arguments.
Update:
@Stephen Rauch’s answer answers this question. However, I don’t recommend using the approach I ask for here. My feature request is intentionally not implemented in Click, since it can result in unexpected behaviors. Click’s recommended approach is to use multiple=True:
@click.option('-u', '--user', 'users', multiple=True)
And in the command line, it will look like:
foo -u alice -u bob -u charlie --bar baz
One way to approach what you are after is to inherit from click.Option, and customize the parser.
Custom Class:
import click
class OptionEatAll(click.Option):
def __init__(self, *args, **kwargs):
self.save_other_options = kwargs.pop('save_other_options', True)
nargs = kwargs.pop('nargs', -1)
assert nargs == -1, 'nargs, if set, must be -1 not {}'.format(nargs)
super(OptionEatAll, self).__init__(*args, **kwargs)
self._previous_parser_process = None
self._eat_all_parser = None
def add_to_parser(self, parser, ctx):
def parser_process(value, state):
# method to hook to the parser.process
done = False
value = [value]
if self.save_other_options:
# grab everything up to the next option
while state.rargs and not done:
for prefix in self._eat_all_parser.prefixes:
if state.rargs[0].startswith(prefix):
done = True
if not done:
value.append(state.rargs.pop(0))
else:
# grab everything remaining
value += state.rargs
state.rargs[:] = []
value = tuple(value)
# call the actual process
self._previous_parser_process(value, state)
retval = super(OptionEatAll, self).add_to_parser(parser, ctx)
for name in self.opts:
our_parser = parser._long_opt.get(name) or parser._short_opt.get(name)
if our_parser:
self._eat_all_parser = our_parser
self._previous_parser_process = our_parser.process
our_parser.process = parser_process
break
return retval
Using Custom Class:
To use the custom class, pass the cls parameter to @click.option() decorator like:
@click.option("--an_option", cls=OptionEatAll)
or if it is desired that the option will eat the entire rest of the command line, not respecting other options:
@click.option("--an_option", cls=OptionEatAll, save_other_options=False)
How does this work?
This works because click is a well designed OO framework. The @click.option() decorator usually instantiates a
click.Option object but allows this behavior to be over ridden with the cls parameter. So it is a relatively
easy matter to inherit from click.Option in our own class and over ride the desired methods.
In this case we over ride click.Option.add_to_parser() and the monkey patch the parser so that we can
eat more than one token if desired.
Test Code:
@click.command()
@click.option('-g', 'greedy', cls=OptionEatAll, save_other_options=False)
@click.option('--polite', cls=OptionEatAll)
@click.option('--other')
def foo(polite, greedy, other):
click.echo('greedy: {}'.format(greedy))
click.echo('polite: {}'.format(polite))
click.echo('other: {}'.format(other))
if __name__ == "__main__":
commands = (
'-g a b --polite x',
'-g a --polite x y --other o',
'--polite x y --other o',
'--polite x -g a b c --other o',
'--polite x --other o -g a b c',
'-g a b c',
'-g a',
'-g',
'extra',
'--help',
)
import sys, time
time.sleep(1)
print('Click Version: {}'.format(click.__version__))
print('Python Version: {}'.format(sys.version))
for cmd in commands:
try:
time.sleep(0.1)
print('-----------')
print('> ' + cmd)
time.sleep(0.1)
foo(cmd.split())
except BaseException as exc:
if str(exc) != '0' and
not isinstance(exc, (click.ClickException, SystemExit)):
raise
Test Results:
Click Version: 6.7
Python Version: 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
-----------
> -g a b --polite x
greedy: ('a', 'b', '--polite', 'x')
polite: None
other: None
-----------
> -g a --polite x y --other o
greedy: ('a', '--polite', 'x', 'y', '--other', 'o')
polite: None
other: None
-----------
> --polite x y --other o
greedy: None
polite: ('x', 'y')
other: o
-----------
> --polite x -g a b c --other o
greedy: ('a', 'b', 'c', '--other', 'o')
polite: ('x',)
other: None
-----------
> --polite x --other o -g a b c
greedy: ('a', 'b', 'c')
polite: ('x',)
other: o
-----------
> -g a b c
greedy: ('a', 'b', 'c')
polite: None
other: None
-----------
> -g a
greedy: ('a',)
polite: None
other: None
-----------
> -g
Error: -g option requires an argument
-----------
> extra
Usage: test.py [OPTIONS]
Error: Got unexpected extra argument (extra)
-----------
> --help
Usage: test.py [OPTIONS]
Options:
-g TEXT
--polite TEXT
--other TEXT
--help Show this message and exit.
You can use this trick.
import click
@click.command()
@click.option('--users', nargs=0, required=True)
@click.argument('users', nargs=-1)
@click.option('--bar')
def fancy_command(users, bar):
users_str = ', '.join(users)
print('Users: {}. Bar: {}'.format(users_str, bar))
if __name__ == '__main__':
fancy_command()
Add fake option with a needed name and none arguments nargs=0, then add ‘argument’ with the unlimited args nargs=-1.
$ python foo --users alice bob charlie --bar baz
Users: alice, bob, charlie. Bar: baz
But be careful with the further options:
$ python foo --users alice bob charlie --bar baz faz
Users: alice, bob, charlie, faz. Bar: baz
I ran into the same issue. Instead of implementing a single command line option with n number of arguments, I decided to use multiple of the same command line option and just letting Click make a tuple out of the arguments under the hood. I ultimately figured if Click didn’t support it, that decision was probably made for a good reason.
https://click.palletsprojects.com/en/7.x/options/#multiple-options
here is an example of what I am saying:
instead of passing a single string argument a splitting on a delimiter:
commit -m foo:bar:baz
I opted to use this:
commit -m foo -m bar -m baz
here is the source code:
@click.command()
@click.option('--message', '-m', multiple=True)
def commit(message):
click.echo('n'.join(message))
This is more to type, but I do think it makes the CLI more user friendly and robust.
I needed this for myself and thought of settling for the solution provided by @nikita-malovichko, though it is very restrictive, but it didn’t work for me (see my comment to that answer) so came up with the below alternative.
My solution doesn’t directly address the question on how to support nargs=*, but it provided a good alternative for myself so sharing it for the benefit of others.
The idea is to use one option that specifies the expected count for another, i.e., set the nargs count dynamically at runtime. Here is a quick demo:
import click
def with_dynamic_narg(cnt_opt, tgt_opt):
class DynamicNArgSetter(click.Command):
def parse_args(self, ctx, args):
ctx.resilient_parsing = True
parser = self.make_parser(ctx)
opts, _, _ = parser.parse_args(args=list(args))
for p in self.params:
if isinstance(p, click.Option) and p.name == tgt_opt:
p.nargs = int(opts[cnt_opt])
ctx.resilient_parsing = False
return super().parse_args(ctx, args)
return DynamicNArgSetter
@click.command(cls=with_dynamic_narg('c', 'n'))
@click.option("-c", type=click.INT)
@click.option("-n", nargs=0)
def f(c, n):
print(c, n)
if __name__ == '__main__':
f()
In the above code, a custom Command class is created that knows the link between the "count" arg and the target arg that takes multiple args. It first does a local parsing in "resilient" mode to detect the count, then uses the count to update the nargs value of the target arg and then resumes parsing in normal mode.
Here is some sample interaction:
$ python t.py -c 0
0 None
$ python t.py -c 1
Usage: t.py [OPTIONS]
Try 't.py --help' for help.
Error: Missing option '-n'.
$ python t.py -c 0 -n a
Usage: t.py [OPTIONS]
Try 't.py --help' for help.
Error: Got unexpected extra argument (a)
$ python t.py -c 1 -n a
1 a
$ python /tmp/t.py -c 2 -n a b
2 ('a', 'b')
Note: The advantage over the official recommendation of using multiple=True is that we can use filename wildcards and let shell expand them. E.g.,
$ touch abc.1 abc.2
$ python t.py -c $(echo abc.* | wc -w) -n abc.*
2 ('abc.1', 'abc.2')