queue.close() "raises" BrokenPipeError

Question:

Using python 3.7 on Unix, the following code:

import multiprocessing

queue = multiprocessing.Queue()
queue.put(1)
# time.sleep(0.01)  Adding this prevents the error
queue.close()

"raises" (in the background) a BrokenPipeError:

Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/queues.py", line 242, in _feed
    send_bytes(obj)
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 404, in _send_bytes
    self._send(header + buf)
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe

To me this clearly looks like a bug, since queue.close() together with queue.join_thread() only exist to avoid that kind of bugs.
Am I missing something?

Notice that the BrokenPipeError is only raised in the background thread which is internally used by python to feed the queue, so from the main process point of view no error is raise and the "only" consequence is just spurious tracebacks printing.

(Also related to multiprocessing.Queue fails intermittently. Bug in Python?)

Asked By: agemO

||

Answers:

This is a bug in python, it happens in 3.7, 3.8 and 3.9, it is resolved in python 3.10, and later versions.

This is a link to the issue on cpython github

Answered By: Ahmed AEK
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.