Is it always safe to modify the `**kwargs` dictionary?

Question

Using the Python function syntax def f(**kwargs), in the function a keyword argument dictionary kwargs is created, and dictionaries are mutable, so the question is, if I modify the kwargs dictionary, is it possible that I might have some effect outside the scope of my function?

From my understanding of how dictionary unpacking and keyword argument packing works, I don’t see any reason to believe it might be unsafe, and it seems to me that there is no danger of this in Python 3.6:

def f(**kwargs):
    kwargs['demo'] = 9

if __name__ == '__main__':
    demo = 4
    f(demo=demo)
    print(demo)     # 4

    kwargs = {}
    f(**kwargs)
    print(kwargs)   # {}

    kwargs['demo'] = 4
    f(**kwargs)
    print(kwargs)    # {'demo': 4}

However, is this implementation-specific, or is it part of the Python spec? Am I overlooking any situation or implementation where (barring modifications to arguments which are themselves mutable, like kwargs['somelist'].append(3)) this sort of modification might be a problem?

Asked By: Paul

||

Source

Answer 1

It is always safe. As the spec says

If the form “**identifier” is present, it is initialized to a new
ordered mapping receiving any excess keyword arguments, defaulting to
a new empty mapping of the same type.

Emphasis added.

You are always guaranteed to get a new mapping-object inside the callable. See this example

def f(**kwargs):
    print((id(kwargs), kwargs))

kwargs = {'foo': 'bar'}
print(id(kwargs))
# 140185018984344
f(**kwargs)
# (140185036822856, {'foo': 'bar'})

So, although f may modify an object that is passed via **, it can’t modify the caller’s **-object itself.

Update: Since you asked about corner cases, here is a special hell for you that does in fact modify the caller’s kwargs:

def f(**kwargs):
    kwargs['recursive!']['recursive!'] = 'Look ma, recursive!'

kwargs = {}
kwargs['recursive!'] = kwargs
f(**kwargs)
assert kwargs['recursive!'] == 'Look ma, recursive!'

This you probably won’t see in the wild, though.

Answered By: user2722968

Answer 2

For Python-level code, the kwargs dict inside a function will always be a new dict.

For C extensions, though, watch out. The C API version of kwargs will sometimes pass a dict through directly. In previous versions, it would even pass dict subclasses through directly, leading to the bug (now fixed) where

'{a}'.format(**collections.defaultdict(int))

would produce '0' instead of raising a KeyError.

If you ever have to write C extensions, possibly including Cython, don’t try to modify the kwargs equivalent, and watch out for dict subclasses on old Python versions.

Answered By: user2357112

Answer 3

Both of above answers are correct in stating that technically, mutating kwargs will never have an effect on the parent scopes.

But… that’s not the end of the story. It is possible for a reference to kwargs to be shared outside of the function scope, and then you run into all the usual shared mutated state problems that you’d expect.

def create_classes(**kwargs):

    class Class1:
        def __init__(self):
            self.options = kwargs

    class Class2:
        def __init__(self):
            self.options = kwargs

    return (Class1, Class2)

Class1, Class2 = create_classes(a=1, b=2)

a = Class1()
b = Class2()

a.options['c'] = 3

print(b.options)
# {'a': 1, 'b': 2, 'c': 3}
# other class's options are mutated because we forgot to copy kwargs

Technically this answers your question, since sharing a reference to mutable kwargs does lead to effects outside of the function scope’s.

I’ve been bitten multiple times by this in production code, and it’s something that I explicitly watch out for now, both in my own code and when reviewing others. The mistake is obvious in my contrived example above, but it’s much sneakier in real code when creating factory funcs that share some common options.

Answered By: Nick Sweeting

Is it always safe to modify the `**kwargs` dictionary?

Question:

Answers: