Why cpython exposes 'PyTuple_SetItem' as C-API if tuple is immutable by design?

Question

Tuple in python is immutable by design, so if we try to mutate a tuple object python emits following TypeError which make sense.

>>> a = (1, 2, 3)
>>> a[0] = 12
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment

So my question is, if tuple is immutable by design why cpython exposes PyTuple_SetItem as C-API?.

From the documentation it’s described as

int PyTuple_SetItem(PyObject *p, Py_ssize_t pos, PyObject *o)

Insert a reference to object o at position pos of the tuple pointed to
by p. Return 0 on success. If pos is out of bounds, return -1 and set
an IndexError exception.

Isn’t this statement exactly equal to tuple[index] = value in python layer?. If the goal was to create a tuple from collection of items we could have use PyTuple_Pack.

Additional note:

After lot of trial and error with ctypes.pythonapi I managed to mutate tuple object using PyTuple_SetItem

import ctypes

from ctypes import py_object

my_tuple = (1, 2, 3)
newObj = py_object(my_tuple)

m = "hello"

# I don't know why I need to Py_DecRef here. 
# Although to reproduce this in your system,  no of times you have 
# to do `Py_DecRef` depends on no of ref count of `newObj` in your system
ctypes.pythonapi.Py_DecRef(newObj)
ctypes.pythonapi.Py_DecRef(newObj)
ctypes.pythonapi.Py_DecRef(newObj)

ctypes.pythonapi.Py_IncRef(m)



PyTuple_SetItem = ctypes.pythonapi.PyTuple_SetItem
PyTuple_SetItem.argtypes = ctypes.py_object, ctypes.c_size_t, ctypes.py_object

PyTuple_SetItem(newObj, 0, m)
print(my_tuple) # this will print `('hello', 2, 3)`

Asked By: Abdul Niyas P M

||

Source

Answer 1

Similarly, there is a PyTuple_Resize function with the warning

Because tuples are supposed to be immutable, this should only be used
if there is only one reference to the object. Do not use this if the
tuple may already be known to some other part of the code. The tuple
will always grow or shrink at the end. Think of this as destroying the
old tuple and creating a new one, only more efficiently.

Looking at the source, there is a guard on the function

if (!PyTuple_Check(op) || Py_REFCNT(op) != 1) {
    .... error ....

Sure enough, this is only allowed when there is only 1 reference to the tuple – that reference being the thing that thinks its a good idea to change it. So, a tuple is "mostly immutable" but C code can change it in limited circumstances to avoid the penalty of creating a new tuple.

Answered By: tdelaney

Answer 2

There is a note in the documentation that say:

Note This function “steals” a reference to o and discards a reference
to an item already in the tuple at the affected position.

Otherwise in Cpython Github docs we can see more details about this function and especially how and why this function stealsthe object reference. We can read:

Incidentally, :c:func:PyTuple_SetItem is the only way to set tuple
items; :c:func:PySequence_SetItem and :c:func:PyObject_SetItem
refuse to do this since tuples are an immutable data type. You should
only use :c:func:PyTuple_SetItem for tuples that you are creating
yourself.

Equivalent code for populating a list can be written using
:c:func:PyList_New and :c:func:PyList_SetItem.

However, in practice, you will rarely use these ways of creating and
populating a tuple or list. There’s a generic function,
:c:func:Py_BuildValue, that can create most common objects from C
values, directed by a :dfn:format string.

Answered By: Chiheb Nexus

Why cpython exposes 'PyTuple_SetItem' as C-API if tuple is immutable by design?

Question:

Answers: