Is there a proper way to handle cursors returned from a postgresql function in psycopg?

Question

I’m trying to make friends with postgresql (14.0 build 1914 64-bit on windows), psycopg2 (2.9.1 installed using pip) and python 3.8.10 on windows.

I have created a postgresql function in a database that returns a cursor, somthing like below

CREATE get_rows 
...
RETURNS refcursor
...
DECLARE
    res1 refcursor;
BEGIN   
    OPEN res1 FOR
        SELECT some_field, and_another_field FROM some_table;

    RETURN res1;
END

The function can be run from pgAdmin4 Quert tool
SELECT get_rows();
and will then return a cursor like "<unnamed portal 1>"

Still within query tool in pgAdmin4 I can issue:

BEGIN;
SELECT get_rows();
FETCH ALL IN "<unnamed portal 2>"; -- Adjust counter number

And this will get me the rows returned by the cursor.

Now I want to replicate this using psycopg instead of pgAdmin4

I have the below code

conn = psycopg2.connect("dbname='" + db_name + "' "
                                "user='" + db_user + "' " +
                                "host='" + db_host + "' "+
                                "password='" + db_passwd + "'")
cursor = conn.cursor()
cursor.callproc('get_rows')
print("cursor.description: ", end = '')
print(cursor.description)
for record in cursor:
    print("record: ", end = '')
    print (record)

The above code only gives the cursor string name (as returned by the postgresql function ‘get_rows’) in the single record of the cursor created by psycopg.

How can I get a cursor-class object from psycopg that provides access the cursor returned by ‘get_rows’?

https://www.psycopg.org/docs/cursor.html says cursor.name is read-only and I dont see an obvious way to connect the cursor from ‘get_rows’ with a psycopg cursor-instance

Asked By: Jon S

||

Source

Answer 1

The cursor link you show refers to the Python DB API cursor not the Postgres one. There is an example of how to do what you want here Server side cursor in section:

Note It is also possible to use a named cursor to consume a cursor created in some other way than using the DECLARE executed by execute(). For example, you may have a PL/pgSQL function returning a cursor:

CREATE FUNCTION reffunc(refcursor) RETURNS refcursor AS $$
BEGIN
    OPEN $1 FOR SELECT col FROM test;
    RETURN $1;
END;
$$ LANGUAGE plpgsql;

You can read the cursor content by calling the function with a regular, non-named, Psycopg cursor:

cur1 = conn.cursor()
cur1.callproc('reffunc', ['curname'])

and then use a named cursor in the same transaction to “steal the cursor”:

cur2 = conn.cursor('curname')
for record in cur2:     # or cur2.fetchone, fetchmany...
    # do something with record
    pass

UPDATE

Be sure and close the named cursor(cur2) to release the server side cursor. So:

cur2.close()

Answered By: Adrian Klaver

Answer 2

Previous answer didn’t work in my case. Below is the one which worked. First stored function (note, there is no refcursor parameter):

CREATE FUNCTION func() RETURNS refcursor AS $$
DECLARE
    ref refcursor;
BEGIN
    OPEN ref FOR SELECT col FROM test;
    RETURN ref;
END;
$$ LANGUAGE plpgsql;

And then you call it as follows (cursor returns a row with refcursor name):

cur = conn.cursor()
cur.callproc("func", [])
ref_cursor_name = cur.fetchone()[0]
ref_cursor = conn.cursor(ref_cursor_name)
data = ref_cursor.fetchall()

Answered By: Dmitry Radchenko

Is there a proper way to handle cursors returned from a postgresql function in psycopg?

Question:

Answers: