Is a variable swap guaranteed to be atomic in python?

Question

With reference to the following link: http://docs.python.org/faq/library.html#what-kinds-of-global-value-mutation-are-thread-safe

I wanted to know if the following:

(x, y) = (y, x)

will be guaranteed atomic in cPython. (x and y are both python variables)

Asked By: dhruvbird

||

Source

Answer 1

~~Yes, yes it will.~~

I stand corrected.

Kragen Sitaker
writes:
Someone recommended using the idiom
spam, eggs = eggs, spam
to get a thread-safe swap. Does this really work? (…)
So if this thread loses control anywhere between the first LOAD_FAST
and the last STORE_FAST, a value could get stored by another thread
into “b” which would then be lost. There isn’t anything keeping this
from happening, is there?
Nope. In general not even a simple
assignment is necessarily thread safe
since performing the assignment may
invoke special methods on an object
which themselves may require a number
of operations. Hopefully the object
will have internally locked its
“state” values, but that’s not always
the case.

But it’s really dictated by what
“thread safety” means in a particular
application, because to my mind there
are many levels of granularity of such
safety so it’s hard to talk about
“thread safety”. About the only thing
the Python interpreter is going to
give you for free is that a built-in
data type should be safe from internal
corruption even with native threading.
In other words if two threads have
a=0xff and a=0xff00, a will end up
with one or the other, but not
accidentally 0xffff as might be
possible in some other languages if a
isn’t protected.

With that said, Python also tends to
execute in such a fashion that you can
get away with an awful lot without
formal locking, if you’re willing to
live on the edge a bit and have
implied dependencies on the actual
objects in use. There was a decent
discussion along those lines here in
c.l.p a while back – search
groups.google.com for the “Critical
sections and mutexes” thread among
others.

Personally, I explicitly lock shared
state (or use constructs designed for
exchanging shared information properly
amongst threads, such as Queue.Queue)
in any multi-threaded application. To
my mind it’s the best protection
against maintenance and evolution down
the road.

—
— David

Answered By: Esteban Küber

Answer 2

Let’s see:

>>> x = 1
>>> y = 2
>>> def swap_xy():
...   global x, y
...   (x, y) = (y, x)
... 
>>> dis.dis(swap_xy)
  3           0 LOAD_GLOBAL              0 (y)
              3 LOAD_GLOBAL              1 (x)
              6 ROT_TWO             
              7 STORE_GLOBAL             1 (x)
             10 STORE_GLOBAL             0 (y)
             13 LOAD_CONST               0 (None)
             16 RETURN_VALUE

It doesn’t appear that they’re atomic: the values of x and y could be changed by another thread between the LOAD_GLOBAL bytecodes, before or after the ROT_TWO, and between the STORE_GLOBAL bytecodes.

If you want to swap two variables atomically, you’ll need a lock or a mutex.

For those desiring empirical proof:

>>> def swap_xy_repeatedly():
...   while 1:
...     swap_xy()
...     if x == y:
...       # If all swaps are atomic, there will never be a time when x == y.
...       # (of course, this depends on "if x == y" being atomic, which it isn't;
...       #  but if "if x == y" isn't atomic, what hope have we for the more complex
...       #  "x, y = y, x"?)
...       print 'non-atomic swap detected'
...       break
... 
>>> t1 = threading.Thread(target=swap_xy_repeatedly)
>>> t2 = threading.Thread(target=swap_xy_repeatedly)
>>> t1.start()
>>> t2.start()
>>> non-atomic swap detected

Answered By: jemfinch

Answer 3

Python atomic for shared data types.

https://sharedatomic.top

The module can be used for atomic operations under multiple processs and multiple threads conditions. High performance python! High concurrency, High performance!

atomic api Example with multiprocessing and multiple threads:

You need the following steps to utilize the module:

create function used by child processes, refer to UIntAPIs, IntAPIs, BytearrayAPIs, StringAPIs, SetAPIs, ListAPIs, in each process, you can create multiple threads.

 def process_run(a):
   def subthread_run(a):
     a.array_sub_and_fetch(b'x0F')

   threadlist = []
   for t in range(5000):
       threadlist.append(Thread(target=subthread_run, args=(a,)))

   for t in range(5000):
       threadlist[t].start()

   for t in range(5000):
       threadlist[t].join()

create the shared bytearray

a = atomic_bytearray(b'ab', length=7, paddingdirection='r', paddingbytes=b'012', mode='m')

start processes / threads to utilize the shared bytearray

 processlist = []

 for p in range(2):

   processlist.append(Process(target=process_run, args=(a,)))

 for p in range(2):

   processlist[p].start()

 for p in range(2):

   processlist[p].join()

 assert a.value == int.to_bytes(27411031864108609, length=8, byteorder='big')

Answered By: Xiquan Ren

Is a variable swap guaranteed to be atomic in python?

Question:

Answers: