Raising exception in init causes SystemError: returned a result with an error set in Python C API

Question

I am using pytest to test my own Python C extension module.
I am trying to check if the TypeError occurs properly when an argument of invalid type is input to the __init__ method.
The method implementation is something like

PyObject * myObject_init(myObject *self, PyObject *args)
{
    if ("# args are invalid")
    {
        PyErr_SetString(PyExc_TypeError, "Invalid Argument");
        return NULL;
    }
}

This makes TypeError occur. But the problem is that when I test this method with pytest like,

def test_init_with_invalid_argument():
    x = "something invalid"
    with pytest.raises(TypeError):
        obj = MyObject(x)

it does fail. The Error message is something like

TypeError: Invalid Argument

The above exception was the direct cause of the following exception:

self = <test_mymodule.TestMyObjectInit object at 0x00000239886D27F0>

    def test_init_with_invalid_argument(self):
        with pytest.raises(TypeError):
>           obj = MyObject(x)
E           SystemError: <class 'mymodule.MyObject'> returned a result with an error set

teststest_init_with_invalid_argument.py:19: SystemError

What is the problem here, and how can I make the test pass?

Asked By: agongji

||

Source

Answer 1

Your __init__ function has the wrong signature.

The __init__ method is defined by the tp_init slot of a PyTypeObject, which if set needs to be an initproc, i.e. a function with the signature

int tp_init(PyObject *self, PyObject *args, PyObject *kwds)

Note that this function returns an int, not a PyObject* like ordinary methods.

The return value should be 0 when initialization succeeds and -1 when initialization fails and an error is set. Note that this is flip-flopped from the behavior of ordinary methods, which return 0 (NULL) on failure and a non-NULL pointer on success. Your function is following the behavior of ordinary methods by returning NULL, but this is the exact opposite of what init needs to do.

Change your init function to return an int, and replace return NULL with return -1. Also, make sure the happy path returns 0 (as opposed to, say, self), and add the missing PyObject *kwds to the argument list.

Answered By: Brian61354270

Answer 2

I write this down just for myself and maybe others.

Python interpreter catches the error when a function which is equivalent to a python method returns NULL. In this case,

PyObject * myObject_init(myObject *self, PyObject *args);

is the method (although the return type is not proper.)

When NULL is returned, python interpreter will raise the error which was set by the C code. In this case,

PyErr_SetString(PyExc_TypeError, "Invalid Argument");

is provoked before returning NULL. So TypeError should be raised.

When you don’t follow the rule of Python-C-API, C code will raise the SystemError for python interpreter to catch. In this case,
myObject_init() function must return int type. If not, The systemError occurs.　Another example is something like below.

int _helper_function(unsigned long k)
{
    int ret;
    if (k is invalid)
    {
        PyErr_SetString(PyExc_TypeError, "Invalid Argument");
        return NULL;
    }
    // some execution
    return ret;
}

This is some helper function which is provoked by another function. It won’t work because this function requires return value as int type although you return NULL. It leads to the SystemError. You have to rewrite this like

int _helper_function(unsigned long k, int * ret)
{
    if (k is invalid)
    {
        PyErr_SetString(PyExc_TypeError, "Invalid Argument");
        return -1;
    }
    // some execution
    *ret = some_value;
    return 0;
}

And after that, validate the return value to decide if you return NULL or not.

Answered By: agongji

Raising exception in init causes SystemError: returned a result with an error set in Python C API

Question:

Answers: