How to tell a function to use the default argument values?

Question:

I have a function foo that calls math.isclose:

import math
def foo(..., rtol=None, atol=None):
    ...
    if math.isclose(x, y, rel_tol=rtol, abs_tol=atol):
        ...
    ...

The above fails in math.isclose if I do not pass rtol and atol to foo:

TypeError: must be real number, not NoneType

I do not want to put the system default argument values in my code (what
if they change down the road?)

Here is what I came up with so far:

import math
def foo(..., rtol=None, atol=None):
    ...
    tols = {}
    if rtol is not None:
        tols["rel_tol"] = rtol
    if atol is not None:
        tols["abs_tol"] = atol
    if math.isclose(x, y, **tols):
        ...
    ...

This looks long and silly and allocates a dict on each invocation of
foo (which calls itself recursively so this is a big deal).

So, what is the best way to tell math.isclose to use the default
tolerances?

PS. There are several related questions. Note that I do not want to know the actual default arguments of math.isclose – all I want it to tell it to use the defaults whatever they are.

Asked By: sds

||

Answers:

One way to do it would be with variadic argument unpacking:

def foo(..., **kwargs):
    ...
    if math.isclose(x, y, **kwargs):
        ...

This would allow you to specify atol and rtol as keyword arguments to the main function foo, which it would then pass on unchanged to math.isclose.

However, I would also say that it is idiomatic that arguments passed to kwargs modify the behaviour of a function in some way other than to be merely passed to sub-functions being called. Therefore, I would suggest that instead, a parameter is named such that it is clear that it will be unpacked and passed unchanged to a sub-function:

def foo(..., isclose_kwargs={}):
    ...
    if math.isclose(x, y, **isclose_kwargs):
        ...

You can see an equivalent pattern in matplotlib (example: plt.subplotssubplot_kw and gridspec_kw, with all other keyword arguments being passed to the Figure constructor as **fig_kw) and seaborn (example: FacetGridsubplot_kws, gridspec_kws).

This is particularly apparent when there are mutiple sub-functions you might want to pass keyword arguments, but retain the default behaviour otherwise:

def foo(..., f1_kwargs={}, f2_kwargs={}, f3_kwargs={}):
    ...
    f1(**f1_kwargs)
    ...
    f2(**f2_kwargs)
    ...
    f3(**f3_kwargs)
    ...

Caveat:

Note that default arguments are only instantiated once, so you should not modify the empty dicts in your function. If there is a need to, you should instead use None as the default argument and instantiate a new empty dict each time the function is run:

def foo(..., isclose_kwargs=None):
    if isclose_kwargs is None:
        isclose_kwargs = {}
    ...
    if math.isclose(x, y, **isclose_kwargs):
        ...

My preference is to avoid this where you know what you’re doing since it is more brief, and in general I don’t like rebinding variables. However, it is definitely a valid idiom, and it can be safer.

Answered By: gmds

Delegate the recursion to a helper function so that you only need to build the dict once.

import math
def foo_helper(..., **kwargs):
    ...
    if math.isclose(x, y, **kwargs):
        ...
    ...


def foo(..., rtol=None, atol=None):
    tols = {}
    if rtol is not None:
        tols["rel_tol"] = rtol
    if atol is not None:
        tols["abs_tol"] = atol

    return foo_helper(x, y, **tols)

Or, instead of passing tolerances to foo, pass a function which already incorporates the desired tolerances.

from functools import partial

# f can take a default value if there is a common set of tolerances
# to use.
def foo(x, y, f):
    ...
    if f(x,y):
        ...
    ...

# Use the defaults
foo(x, y, f=math.isclose)
# Use some other tolerances
foo(x, y, f=partial(math.isclose, rtol=my_rtel))
foo(x, y, f=partial(math.isclose, atol=my_atol))
foo(x, y, f=partial(math.isclose, rtol=my_rtel, atol=my_atol))
Answered By: chepner

There really aren’t many ways to make a function use its default arguments… You only have two options:

  1. Pass the real default values
  2. Don’t pass the arguments at all

Since none of the options are great, I’m going to make an exhaustive list so you can compare them all.

  • Use **kwargs to pass through arguments

    Define your method using **kwargs and pass those to math.isclose:

    def foo(..., **kwargs):
        ...
        if math.isclose(x, y, **kwargs):
    

    Cons:

    • the parameter names of both functions have to match (e.g. foo(1, 2, rtol=3) won’t work)
  • Manually construct a **kwargs dict

    def foo(..., rtol=None, atol=None):
        ...
        kwargs = {}
        if rtol is not None:
            kwargs["rel_tol"] = rtol
        if atol is not None:
            kwargs["abs_tol"] = atol
        if math.isclose(x, y, **kwargs):
    

    Cons:

    • ugly, a pain to code, and not fast
  • Hard-code the default values

    def foo(..., rtol=1e-09, atol=0.0):
        ...
        if math.isclose(x, y, rel_tol=rtol, abs_tol=atol):
    

    Cons:

    • hard-coded values
  • Use introspection to find the default values

    You can use the inspect module to determine the default values at run time:

    import inspect, math
    
    signature = inspect.signature(math.isclose)
    DEFAULT_RTOL = signature.parameters['rel_tol'].default
    DEFAULT_ATOL = signature.parameters['abs_tol'].default
    
    def foo(..., rtol=DEFAULT_RTOL, atol=DEFAULT_ATOL):
        ...
        if math.isclose(x, y, rel_tol=rtol, abs_tol=atol):
    

    Cons:

Answered By: Aran-Fey

The correct solution would be to use the same defaults as math.isclose(). There is no need to hard-code them, as you can get the current defaults with the inspect.signature() function:

import inspect
import math

_isclose_params = inspect.signature(math.isclose).parameters

def foo(..., rtol=_isclose_params['rel_tol'].default, atol=_isclose_params['abs_tol'].default):
    # ...

Quick demo:

>>> import inspect
>>> import math
>>> params = inspect.signature(math.isclose).parameters
>>> params['rel_tol'].default
1e-09
>>> params['abs_tol'].default
0.0

This works because math.isclose() defines its arguments using the Argument Clinic tool:

[T]he original motivation for Argument Clinic was to provide introspection “signatures” for CPython builtins. It used to be, the introspection query functions would throw an exception if you passed in a builtin. With Argument Clinic, that’s a thing of the past!

Under the hood, the math.isclose() signature is actually stored as a string:

>>> math.isclose.__text_signature__
'($module, /, a, b, *, rel_tol=1e-09, abs_tol=0.0)'

This is parsed out by the inspect signature support to give you the actual values.

Not all C-defined functions use Argument Clinic yet, the codebase is being converted on a case-by-case basis. math.isclose() was converted for Python 3.7.0.

You could use the __doc__ string as a fallback, as in earlier versions this too contains the signature:

>>> import math
>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=6, micro=8, releaselevel='final', serial=0)
>>> math.isclose.__doc__.splitlines()[0]
'isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0) -> bool'

so a slightly more generic fallback could be:

import inspect

def func_defaults(f):
    try:
        params = inspect.signature(f).parameters
    except ValueError:
        # parse out the signature from the docstring
        doc = f.__doc__
        first = doc and doc.splitlines()[0]
        if first is None or f.__name__ not in first or '(' not in first:
            return {}
        sig = inspect._signature_fromstr(inspect.Signature, math.isclose, first)
        params = sig.parameters
    return {
        name: p.default for name, p in params.items()
        if p.default is not inspect.Parameter.empty
    }

I’d see this as a stop-gap measure only needed to support older Python 3.x releases. The function produces a dictionary keyed on parameter name:

>>> import sys
>>> import math
>>> sys.version_info
sys.version_info(major=3, minor=6, micro=8, releaselevel='final', serial=0)
>>> func_defaults(math.isclose)
{'rel_tol': 1e-09, 'abs_tol': 0.0}

Note that copying the Python defaults is very low risk; unless there is a bug, the values are not prone to change. So another option could be to hardcode the 3.5 / 3.6 known defaults as a fallback, and use the signature provided in 3.7 and newer:

try:
    # Get defaults through introspection in newer releases
    _isclose_params = inspect.signature(math.isclose).parameters
    _isclose_rel_tol = _isclose_params['rel_tol'].default
    _isclose_abs_tol = _isclose_params['abs_tol'].default
except ValueError:
    # Python 3.5 / 3.6 known defaults
    _isclose_rel_tol = 1e-09
    _isclose_abs_tol = 0.0

Note however that you are at greater risk of not supporting future, additional parameters and defaults. At least the inspect.signature() approach would let you add an assertion about the number of parameters that your code expects there to be.

Answered By: Martijn Pieters
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.