What is the correct way to document a **kwargs parameter?
Question:
I’m using Sphinx and the autodoc extension to generate API documentation for my Python modules. Whilst I can see how to nicely document specific parameters, I cannot find an example of how to document a **kwargs parameter.
Does anyone have a good example of a clear way to document these?
Answers:
I think subprocess-module’s docs is a good example. Give an exhaustive list of all parameters for a top/parent class. Then just refer to that list for all other occurrences of **kwargs.
If anyone else is looking for some valid syntax.. Here’s an example docstring. This is just how I did it, I hope it’s useful to you, but I can’t claim that it’s compliant with anything in particular.
def bar(x=True, y=False):
"""
Just some silly bar function.
:Parameters:
- `x` (`bool`) - dummy description for x
- `y` (`string`) - dummy description for y
:return: (`string`) concatenation of x and y.
"""
return str(x) + y
def foo (a, b, **kwargs):
"""
Do foo on a, b and some other objects.
:Parameters:
- `a` (`int`) - A number.
- `b` (`int`, `string`) - Another number, or maybe a string.
- `**kwargs` - remaining keyword arguments are passed to `bar`
:return: Success
:rtype: `bool`
"""
return len(str(a) + str(b) + bar(**kwargs)) > 20
There is a doctstring example for Sphinx in their documentation. Specifically they show the following:
def public_fn_with_googley_docstring(name, state=None):
"""This function does something.
Args:
name (str): The name to use.
Kwargs:
state (bool): Current state to be in.
Returns:
int. The return code::
0 -- Success!
1 -- No good.
2 -- Try again.
Raises:
AttributeError, KeyError
A really great idea. A way you might use me is
>>> print public_fn_with_googley_docstring(name='foo', state=None)
0
BTW, this always returns 0. **NEVER** use with :class:`MyPublicClass`.
"""
return 0
Though you asked about sphinx explicitly, I would also point to the Google Python Style Guide. Their docstring example seems to imply that they don’t call out kwargs specifically. (other_silly_variable=None)
def fetch_bigtable_rows(big_table, keys, other_silly_variable=None):
"""Fetches rows from a Bigtable.
Retrieves rows pertaining to the given keys from the Table instance
represented by big_table. Silly things may happen if
other_silly_variable is not None.
Args:
big_table: An open Bigtable Table instance.
keys: A sequence of strings representing the key of each table row
to fetch.
other_silly_variable: Another optional variable, that has a much
longer name than the other args, and which does nothing.
Returns:
A dict mapping keys to the corresponding table row data
fetched. Each row is represented as a tuple of strings. For
example:
{'Serak': ('Rigel VII', 'Preparer'),
'Zim': ('Irk', 'Invader'),
'Lrrr': ('Omicron Persei 8', 'Emperor')}
If a key from the keys argument is missing from the dictionary,
then that row was not found in the table.
Raises:
IOError: An error occurred accessing the bigtable.Table object.
"""
pass
A-B-B has a question about the accepted answer of referencing the subprocess management documentation. If you import a module, you can quickly see the module docstrings via inspect.getsource.
An example from the python interpreter using Silent Ghost’s recommendation:
>>> import subprocess
>>> import inspect
>>> import print inspect.getsource(subprocess)
Of course you can also view the module documentation via help function. For example help(subprocess)
I’m not personally a fan of the subprocess docstring for kwargs as an example, but like the Google example it doesn’t list kwargs seperately as shown in the Sphinx documentation example.
def call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete, then
return the returncode attribute.
The arguments are the same as for the Popen constructor. Example:
retcode = call(["ls", "-l"])
"""
return Popen(*popenargs, **kwargs).wait()
I’m including this answer to A-B-B’s question because it’s worth noting that you can review any module’s source or documentation this way for insights and inspiration for commenting your code.
After finding this question I settled on the following, which is valid Sphinx and works fairly well:
def some_function(first, second="two", **kwargs):
r"""Fetches and returns this thing
:param first:
The first parameter
:type first: ``int``
:param second:
The second parameter
:type second: ``str``
:param **kwargs:
See below
:Keyword Arguments:
* *extra* (``list``) --
Extra stuff
* *supplement* (``dict``) --
Additional content
"""
The r"""...""" is required to make this a “raw” docstring and thus keep the * intact (for Sphinx to pick up as a literal * and not the start of “emphasis”).
The chosen formatting (bulleted list with parenthesized type and m-dash-separated description) is simply to match the automated formatting provided by Sphinx.
Once you’ve gone to this effort of making the “Keyword Arguments” section look like the default “Parameters” section, it seems like it might be easier to roll your own parameters section from the outset (as per some of the other answers), but as a proof of concept this is one way to achieve a nice look for supplementary **kwargs if you’re already using Sphinx.
Google Style docstrings parsed by Sphinx
Disclaimer: not tested.
From this cutout of the sphinx docstring example, the *args and **kwargs are left unexpanded:
def module_level_function(param1, *args, param2=None, **kwargs):
"""
...
Args:
param1 (int): The first parameter.
param2 (Optional[str]): The second parameter. Defaults to None.
Second line of description should be indented.
*args: Variable length argument list.
**kwargs: Arbitrary keyword arguments.
I would suggest the following solution for compactness:
"""
Args:
param1 (int): The first parameter.
param2 (Optional[str]): The second parameter. Defaults to None.
Second line of description should be indented.
*param3 (int): description
*param4 (str):
...
**key1 (int): description
**key2 (int): description
...
Notice how, Optional is not required for **key arguments.
Otherwise, you can try to explicitly list the *args under Other Parameters and **kwargs under the Keyword Args (see docstring sections):
"""
Args:
param1 (int): The first parameter.
param2 (Optional[str]): The second parameter. Defaults to None.
Second line of description should be indented.
Other Parameters:
param3 (int): description
param4 (str):
...
Keyword Args:
key1 (int): description
key2 (int): description
...
This depends on the style of documentation you use, but if you are using the numpydoc style it is recommend to document **kwargs using Other Parameters.
For example, following quornian’s example:
def some_function(first, second="two", **kwargs):
"""Fetches and returns this thing
Parameters
----------
first : `int`
The first parameter
second : `str`, optional
The second parameter
Other Parameters
----------------
extra : `list`, optional
Extra stuff. Default ``[]``.
suplement : `dict`, optional
Additional content. Default ``{'key' : 42}``.
"""
Note especially that it is recommended to give the defaults of kwargs, since these are not obvious from the function signature.
If you are looking for how to do this in numpydoc style, you can simply mention **kwargs in Parameters section without specifying type – as demonstrated in numpydoc example from the sphinx extension napolean and docstring guide from pandas documentation sprint 2018.
Here’s an example I found from LSST developer guide which very well explains what should be the description of **kwargs parameter:
def demoFunction(namedArg, *args, flag=False, **kwargs):
"""Demonstrate documentation for additional keyword and
positional arguments.
Parameters
----------
namedArg : `str`
A named argument that is documented like always.
*args : `str`
Additional names.
Notice how the type is singular since the user is expected to pass individual
`str` arguments, even though the function itself sees ``args`` as an iterable
of `str` objects).
flag : `bool`
A regular keyword argument.
**kwargs
Additional keyword arguments passed to `otherApi`.
Usually kwargs are used to pass parameters to other functions and
methods. If that is the case, be sure to mention (and link) the
API or APIs that receive the keyword arguments.
If kwargs are being used to generate a `dict`, use the description to
document the use of the keys and the types of the values.
"""
Alternatively, building upon what @Jonas Adler suggested, I find it better to put the **kwargs and its description in Other Parameters section – even this example from matplotlib documentation guide suggests the same.
I am unable to find the actual link to the docs but this works (using Sphinx 3.4.3):
class Foo:
"""A Foo implementation
:param str foo: Foo
:param int bar: Bar
:keyword str key1: kwarg 1
:keyword str key2: kwarg 2
:keyword int key3: kwarg 3
"""
def __init__(self, foo, bar, **kwargs):
pass
I’m using Sphinx and the autodoc extension to generate API documentation for my Python modules. Whilst I can see how to nicely document specific parameters, I cannot find an example of how to document a **kwargs parameter.
Does anyone have a good example of a clear way to document these?
I think subprocess-module’s docs is a good example. Give an exhaustive list of all parameters for a top/parent class. Then just refer to that list for all other occurrences of **kwargs.
If anyone else is looking for some valid syntax.. Here’s an example docstring. This is just how I did it, I hope it’s useful to you, but I can’t claim that it’s compliant with anything in particular.
def bar(x=True, y=False):
"""
Just some silly bar function.
:Parameters:
- `x` (`bool`) - dummy description for x
- `y` (`string`) - dummy description for y
:return: (`string`) concatenation of x and y.
"""
return str(x) + y
def foo (a, b, **kwargs):
"""
Do foo on a, b and some other objects.
:Parameters:
- `a` (`int`) - A number.
- `b` (`int`, `string`) - Another number, or maybe a string.
- `**kwargs` - remaining keyword arguments are passed to `bar`
:return: Success
:rtype: `bool`
"""
return len(str(a) + str(b) + bar(**kwargs)) > 20
There is a doctstring example for Sphinx in their documentation. Specifically they show the following:
def public_fn_with_googley_docstring(name, state=None):
"""This function does something.
Args:
name (str): The name to use.
Kwargs:
state (bool): Current state to be in.
Returns:
int. The return code::
0 -- Success!
1 -- No good.
2 -- Try again.
Raises:
AttributeError, KeyError
A really great idea. A way you might use me is
>>> print public_fn_with_googley_docstring(name='foo', state=None)
0
BTW, this always returns 0. **NEVER** use with :class:`MyPublicClass`.
"""
return 0
Though you asked about sphinx explicitly, I would also point to the Google Python Style Guide. Their docstring example seems to imply that they don’t call out kwargs specifically. (other_silly_variable=None)
def fetch_bigtable_rows(big_table, keys, other_silly_variable=None):
"""Fetches rows from a Bigtable.
Retrieves rows pertaining to the given keys from the Table instance
represented by big_table. Silly things may happen if
other_silly_variable is not None.
Args:
big_table: An open Bigtable Table instance.
keys: A sequence of strings representing the key of each table row
to fetch.
other_silly_variable: Another optional variable, that has a much
longer name than the other args, and which does nothing.
Returns:
A dict mapping keys to the corresponding table row data
fetched. Each row is represented as a tuple of strings. For
example:
{'Serak': ('Rigel VII', 'Preparer'),
'Zim': ('Irk', 'Invader'),
'Lrrr': ('Omicron Persei 8', 'Emperor')}
If a key from the keys argument is missing from the dictionary,
then that row was not found in the table.
Raises:
IOError: An error occurred accessing the bigtable.Table object.
"""
pass
A-B-B has a question about the accepted answer of referencing the subprocess management documentation. If you import a module, you can quickly see the module docstrings via inspect.getsource.
An example from the python interpreter using Silent Ghost’s recommendation:
>>> import subprocess
>>> import inspect
>>> import print inspect.getsource(subprocess)
Of course you can also view the module documentation via help function. For example help(subprocess)
I’m not personally a fan of the subprocess docstring for kwargs as an example, but like the Google example it doesn’t list kwargs seperately as shown in the Sphinx documentation example.
def call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete, then
return the returncode attribute.
The arguments are the same as for the Popen constructor. Example:
retcode = call(["ls", "-l"])
"""
return Popen(*popenargs, **kwargs).wait()
I’m including this answer to A-B-B’s question because it’s worth noting that you can review any module’s source or documentation this way for insights and inspiration for commenting your code.
After finding this question I settled on the following, which is valid Sphinx and works fairly well:
def some_function(first, second="two", **kwargs):
r"""Fetches and returns this thing
:param first:
The first parameter
:type first: ``int``
:param second:
The second parameter
:type second: ``str``
:param **kwargs:
See below
:Keyword Arguments:
* *extra* (``list``) --
Extra stuff
* *supplement* (``dict``) --
Additional content
"""
The r"""...""" is required to make this a “raw” docstring and thus keep the * intact (for Sphinx to pick up as a literal * and not the start of “emphasis”).
The chosen formatting (bulleted list with parenthesized type and m-dash-separated description) is simply to match the automated formatting provided by Sphinx.
Once you’ve gone to this effort of making the “Keyword Arguments” section look like the default “Parameters” section, it seems like it might be easier to roll your own parameters section from the outset (as per some of the other answers), but as a proof of concept this is one way to achieve a nice look for supplementary **kwargs if you’re already using Sphinx.
Google Style docstrings parsed by Sphinx
Disclaimer: not tested.
From this cutout of the sphinx docstring example, the *args and **kwargs are left unexpanded:
def module_level_function(param1, *args, param2=None, **kwargs):
"""
...
Args:
param1 (int): The first parameter.
param2 (Optional[str]): The second parameter. Defaults to None.
Second line of description should be indented.
*args: Variable length argument list.
**kwargs: Arbitrary keyword arguments.
I would suggest the following solution for compactness:
"""
Args:
param1 (int): The first parameter.
param2 (Optional[str]): The second parameter. Defaults to None.
Second line of description should be indented.
*param3 (int): description
*param4 (str):
...
**key1 (int): description
**key2 (int): description
...
Notice how, Optional is not required for **key arguments.
Otherwise, you can try to explicitly list the *args under Other Parameters and **kwargs under the Keyword Args (see docstring sections):
"""
Args:
param1 (int): The first parameter.
param2 (Optional[str]): The second parameter. Defaults to None.
Second line of description should be indented.
Other Parameters:
param3 (int): description
param4 (str):
...
Keyword Args:
key1 (int): description
key2 (int): description
...
This depends on the style of documentation you use, but if you are using the numpydoc style it is recommend to document **kwargs using Other Parameters.
For example, following quornian’s example:
def some_function(first, second="two", **kwargs):
"""Fetches and returns this thing
Parameters
----------
first : `int`
The first parameter
second : `str`, optional
The second parameter
Other Parameters
----------------
extra : `list`, optional
Extra stuff. Default ``[]``.
suplement : `dict`, optional
Additional content. Default ``{'key' : 42}``.
"""
Note especially that it is recommended to give the defaults of kwargs, since these are not obvious from the function signature.
If you are looking for how to do this in numpydoc style, you can simply mention **kwargs in Parameters section without specifying type – as demonstrated in numpydoc example from the sphinx extension napolean and docstring guide from pandas documentation sprint 2018.
Here’s an example I found from LSST developer guide which very well explains what should be the description of **kwargs parameter:
def demoFunction(namedArg, *args, flag=False, **kwargs):
"""Demonstrate documentation for additional keyword and
positional arguments.
Parameters
----------
namedArg : `str`
A named argument that is documented like always.
*args : `str`
Additional names.
Notice how the type is singular since the user is expected to pass individual
`str` arguments, even though the function itself sees ``args`` as an iterable
of `str` objects).
flag : `bool`
A regular keyword argument.
**kwargs
Additional keyword arguments passed to `otherApi`.
Usually kwargs are used to pass parameters to other functions and
methods. If that is the case, be sure to mention (and link) the
API or APIs that receive the keyword arguments.
If kwargs are being used to generate a `dict`, use the description to
document the use of the keys and the types of the values.
"""
Alternatively, building upon what @Jonas Adler suggested, I find it better to put the **kwargs and its description in Other Parameters section – even this example from matplotlib documentation guide suggests the same.
I am unable to find the actual link to the docs but this works (using Sphinx 3.4.3):
class Foo:
"""A Foo implementation
:param str foo: Foo
:param int bar: Bar
:keyword str key1: kwarg 1
:keyword str key2: kwarg 2
:keyword int key3: kwarg 3
"""
def __init__(self, foo, bar, **kwargs):
pass