A memory leak in a simple Python C-extension

Question:

I have some code similar to the one below. That code leaks, and I don’t know why. The thing that leaks is a simple creation of a Python class’ instance inside a C code. The function I use to check the leak is create_n_times that’s defined below and just creates new Python instances and derefrences them in a loop.

This is not an MWE per-se, but part of an example. To make it easier to understand, what the code does is:

  1. The Python code defines the dataclass and registers it into the C-extension using set_ip_settings_type.
  2. Then, a C-extension function create_n_times is called and that function creates and destroys n instances of the Python dataclass.

Can anyone help?

In Python:

import c_api

@dataclass
class IpSettings:
    ip: str
    port: int
    dhcp: bool

c_api.set_ip_settings_type(IpSettings)
c_api.generate_n_times(100000)

In C++ I have the following code that’s compiled into a Python extension called c_api (it’s a part of that library’s definition):

#include <Python.h>

// ... Other functions including a "PyInit" function

extern "C" {
    
PyObject* ip_settings_type = NULL;
PyObject* set_ip_settings_type(PyObject* tp)
{
    Py_XDECREF(ip_settings_type);
    Py_INCREF(tp);
    ip_settings_type = tp;
    return Py_None;
}

PyObject* create_n_times(PyObject* n)
{
   long n_ = PyLong_AsLong(n);
   for (int i = 0; i < n_ ++i)
   {
       PyObject* factory_object = ip_settings_type;

       PyObject* args = PyTuple_New(3);
       PyTuple_SetItem(args, 0, PyUnicode_FromString("123.123.123.123"));
       PyTuple_SetItem(args, 1, PyLong_FromUnsignedLong(1231));
       PyTuple_SetItem(args, 2, Py_False);

       PyObject* obj = PyObject_CallObject(factory_object, args);
       Py_DECREF(obj);
   }

    return Py_None;
}

}
Asked By: EZLearner

||

Answers:

PyTuple_SetItem steals the reference to the supplied object, but Py_False is a single object. When the args tuple is destroyed, the reference count for Py_False isgetting mangled.

Use PyBool_FromLong(0) to create a new reference to Py_False, like the other two calls to PyTuple_SetItem. (see docs.python.org/3/c-api/bool.html)

Answered By: Neil
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.