Thread Safety in Python's dictionary

Question

I have a class which holds a dictionary

class OrderBook:
    orders = {'Restaurant1': None,
              'Restaurant2': None,
              'Restaurant3': None,
              'Restaurant4': None}

    @staticmethod
    def addOrder(restaurant_name, orders):
        OrderBook.orders[restaurant_name] = orders

And I am running 4 threads (one for each restaurant) that call the method OrderBook.addOrder. Here is the function ran by each thread:

def addOrders(restaurant_name):

    #creates orders
    ...

    OrderBook.addOrder(restaurant_name, orders)

Is this safe, or do I have to use a lock before calling addOrder?

Asked By: nmat

||

Source

Answer 1

Yes, built-in types are inherently thread-safe:
http://docs.python.org/glossary.html#term-global-interpreter-lock

This simplifies the CPython implementation by making the object model (including critical built-in types such as dict) implicitly safe against concurrent access.

Answered By: user626998

Answer 2

Python’s built-in structures are thread-safe for single operations, but it can sometimes be hard to see where a statement really becomes multiple operations.

Your code should be safe. Keep in mind: a lock here will add almost no overhead, and will give you peace of mind.

https://web.archive.org/web/20201108091210/http://effbot.org/pyfaq/what-kinds-of-global-value-mutation-are-thread-safe.htm has more details.

Answered By: Ned Batchelder

Answer 3

Google’s style guide advises against relying on dict atomicity

This is explained in further detail at: Is Python variable assignment atomic?

Do not rely on the atomicity of built-in types.

While Python’s built-in data types such as dictionaries appear to have atomic operations, there are corner cases where they aren’t atomic (e.g. if __hash__ or __eq__ are implemented as Python methods) and their atomicity should not be relied upon. Neither should you rely on atomic variable assignment (since this in turn depends on dictionaries).

Use the Queue module’s Queue data type as the preferred way to communicate data between threads. Otherwise, use the threading module and its locking primitives. Learn about the proper use of condition variables so you can use threading.Condition instead of using lower-level locks.

And I agree with this one: there is already the GIL in CPython, so the performance hit of using a Lock will be negligible. Much more costly will be the hours spent bug hunting in a complex codebase when those CPython implementation details change one day.

Answered By: Ciro Santilli OurBigBook.com

Answer 4

When using python’s builtin dict, set and get are atomic (because of
cpython’s GIL).However, it seems like bad practice though since operations such as .items are not atomic.

Note – get-add-set operations are not thread-safe if multiple threads are working on the same dict keys.

Answered By: ravi malhotra

Answer 5

dict() is not thread-safe. You just need to execute the following code to see a runtime error.

import threading

dic = dict()

def producer():
    for i in range(1000000):
        dic[i] = i

def consumer():
    for key, value in dic.items():
        print(key, value)

th1 = threading.Thread(target=producer)
th2 = threading.Thread(target=consumer)
th1.start()
th2.start()
th1.join()
th2.join()

RuntimeError: dictionary changed size during iteration

Answered By: H.H

Thread Safety in Python's dictionary

Question:

Answers: