With assignment expressions in Python 3.8, why do we need to use `as` in `with`?

Question

Now that PEP 572 has been accepted, Python 3.8 is destined to have assignment expressions, so we can use an assignment expression in with, i.e.

with (f := open('file.txt')):
    for l in f:
        print(f)

instead of

with open('file.txt') as f:
    for l in f:
        print(f)

and it would work as before.

What use does the as keyword have with the with statement in Python 3.8? Isn’t this against the Zen of Python: “There should be one — and preferably only one — obvious way to do it.”?

When the feature was originally proposed, it wasn’t clearly specified whether the assignment expression should be parenthesized in with and that

with f := open('file.txt'):
    for l in f:
        print(f)

could work. However, in Python 3.8a0,

with f := open('file.txt'):
    for l in f:
        print(f)

will cause

  File "<stdin>", line 1
    with f := open('file.txt'):
           ^
SyntaxError: invalid syntax

but the parenthesized expression works.

Asked By: Antti Haapala — Слава Україні

||

Source

Answer 1

TL;DR: The behaviour is not the same for both constructs, even though there wouldn’t be discernible differences between the 2 examples.

You should almost never need := in a with statement, and sometimes it is very wrong. When in doubt, always use with ... as ... when you need the managed object within the with block.

In with context_manager as managed, managed is bound to the return value of context_manager.__enter__(), whereas in with (managed := context_manager), managed is bound to the context_manager itself and the return value of the __enter__() method call is discarded. The behaviour is almost identical for open files, because their __enter__ method returns self.

The first excerpt is roughly analogous to

_mgr = (f := open('file.txt')) # `f` is assigned here, even if `__enter__` fails
_mgr.__enter__()               # the return value is discarded

exc = True
try:
    try:
        BLOCK
    except:
        # The exceptional case is handled here
        exc = False
        if not _mgr.__exit__(*sys.exc_info()):
            raise
        # The exception is swallowed if exit() returns true
finally:
    # The normal and non-local-goto cases are handled here
    if exc:
        _mgr.__exit__(None, None, None)

whereas the as form would be

_mgr = open('file.txt')   # 
_value = _mgr.__enter__() # the return value is kept

exc = True
try:
    try:
        f = _value        # here f is bound to the return value of __enter__
                          # and therefore only when __enter__ succeeded
        BLOCK
    except:
        # The exceptional case is handled here
        exc = False
        if not _mgr.__exit__(*sys.exc_info()):
            raise
        # The exception is swallowed if exit() returns true
finally:
    # The normal and non-local-goto cases are handled here
    if exc:
        _mgr.__exit__(None, None, None)

i.e. with (f := open(...)) would set f to the return value of open, whereas with open(...) as f binds f to the return value of the implicit __enter__() method call.

Now, in case of files and streams, file.__enter__() will return self if it succeeds, so the behaviour for these two approaches is almost the same – the only difference is in the event that __enter__ throws an exception.

The fact that assignment expressions will often work instead of as is deceptive, because there are many classes where _mgr.__enter__() returns an object that is distinct from self. In that case an assignment expression works differently: the context manager is assigned, instead of the managed object. For example unittest.mock.patch is a context manager that will return the mock object. The documentation for it has the following example:

>>> thing = object()
>>> with patch('__main__.thing', new_callable=NonCallableMock) as mock_thing:
...     assert thing is mock_thing
...     thing()
...
Traceback (most recent call last):
  ...
TypeError: 'NonCallableMock' object is not callable

Now, if it were to be written to use an assignment expression, the behaviour would be different:

>>> thing = object()
>>> with (mock_thing := patch('__main__.thing', new_callable=NonCallableMock)):
...     assert thing is mock_thing
...     thing()
...
Traceback (most recent call last):
  ...
AssertionError
>>> thing
<object object at 0x7f4aeb1ab1a0>
>>> mock_thing
<unittest.mock._patch object at 0x7f4ae910eeb8>

mock_thing is now bound to the context manager instead of the new mock object.

Answered By: Antti Haapala — Слава Україні

With assignment expressions in Python 3.8, why do we need to use `as` in `with`?

Question:

Answers: