psycopg2.InterfaceError: connection already closed / pgr_astar

Question

I am using psycopg2 to access a postgresql database from python. When I try to run the pgrouting function “pgr_astar” for the shortest path, I receive an error

cur = db.cursor()
psycopg2.InterfaceError: connection already closed

Basically, what happened is: when pgr_astar does not find a path between two points, it creates a crash of the database, and it closes the connection to the database.

How is it possible to avoid this?

I tried to isolate the function with a try/except and to create its own connection, so no problem if it closes:

conn = psycopg2.connect(...)
try:
    result = getRecords(conn,query_pgr_astar)
    return float(result[0]['sum'])
except:
    return 500000000.0
conn.close()

But in this case I receive a

cur.execute(query)
psycopg2.DatabaseError: SSL SYSCALL error: EOF detected

How is it possible to manage impossible paths?

(I did the same with pgdb and the old shortest_path_astar function (not available to me anymore), and I had no problem)

Postgresql log is:

2013-11-27 15:54:35 CET LOG:  terminating any other active server processes
2013-11-27 15:54:35 CET WARNING:  terminating connection because of crash of another server process
2013-11-27 15:54:35 CET DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2013-11-27 15:54:35 CET HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2013-11-27 15:54:35 CET WARNING:  terminating connection because of crash of another server process
2013-11-27 15:54:35 CET DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2013-11-27 15:54:35 CET HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2013-11-27 15:54:35 CET LOG:  all server processes terminated; reinitializing
2013-11-27 15:54:35 CET LOG:  database system was interrupted; last known up at 2013-11-27 15:42:01 CET
2013-11-27 15:54:35 CET LOG:  database system was not properly shut down; automatic recovery in progress
2013-11-27 15:54:35 CET LOG:  record with zero length at 65/80875D60
2013-11-27 15:54:35 CET LOG:  redo is not required
2013-11-27 15:54:35 CET LOG:  autovacuum launcher started
2013-11-27 15:54:35 CET LOG:  database system is ready to accept connections

The query I do is:

SELECT SUM(t2.length) FROM (SELECT id2 FROM pgr_astar('SELECT edge_id AS id, vertex_id1 AS source, vertex_id2 AS target, ' || '(CASE WHEN door = ''S'' THEN -1.0  ELSE  (length * (CASE network WHEN ''0'' THEN 1.0  WHEN ''10'' THEN 1.2  WHEN ''20'' THEN 1.5  WHEN ''30'' THEN 2.0  ELSE 1.0  END) *  (CASE type WHEN ''9.1'' THEN CASE WHEN level1 < level2 THEN 3.0 ELSE 2.0 END  WHEN ''9.2'' THEN CASE WHEN level1 < level2 THEN 15.0 ELSE 12.0 END  ELSE 1.0  END) +  (CASE type WHEN ''9.3'' THEN 40.0  ELSE 0.0  END)  )  END)::float8 AS cost, ' || '(CASE WHEN door_rev = ''S'' THEN -1.0  ELSE  (length * (CASE network WHEN ''0'' THEN 1.0  WHEN ''10'' THEN 1.2  WHEN ''20'' THEN 1.5  WHEN ''30'' THEN 2.0  ELSE 1.0  END) *  (CASE type WHEN ''9.1'' THEN CASE WHEN level1 < level2 THEN 3.0 ELSE 2.0 END  WHEN ''9.2'' THEN CASE WHEN level1 < level2 THEN 15.0 ELSE 12.0 END  ELSE 1.0  END) +  (CASE type WHEN ''9.3'' THEN 40.0  ELSE 0.0  END)  )  END )::float8 AS reverse_cost, ' || 'x1, y1, x2, y2 FROM edges', " + str(destination_route) + ", " + str(origin_route) + ", TRUE, TRUE)) as t1, edges as t2 where t1.id2 = t2.edge_id

door = ”S” means door is closed, and in this case, value is -1.

Asked By: Antonin

||

Source

Answer 1

I assume you can check your connection status in python (I don’t use python) so you can recover and do something sensible, like reconnect. You might need to sleep for a couple of seconds to let the database recover from the crash before you reconnect.

Regarding the pgrouting issue, I would like to see a simple test case that can reproduce this problem in pgadmin or psql shell, filed as a bug so I can look into it. Our code should never crash the server, but sometimes it does and we need to fix those issues.

Answered By: Stephen Woodbridge

Answer 2

I had the same errors, this happens when the posqresql server cuts the connection without warning. On my side the posgresql server was doing oom-kill. The ideal is to understand why the servers kill the connection and to solve this problem. And you can all expect this error so that psycopg2 tries to reconnect automatically. If you go through a haproxy to share the connection between different db endpoints, it will simply change nodes, otherwise it is to be hoped that the server will be back soon I guess.

     cursor = connection.cursor()
InterfaceError: connection already closed

Except and reconnect :

try:
    cursor = conn.cursor()
except psycopg2.InterfaceError as e:
    print('{} - connection will be reset'.format(e))
    # Close old connection 
    if conn:
        if cursor:
            cursor.close()
        conn.close()
    conn = None
    cursor = None
    
    # Reconnect 
    conn = psycopg2.connect(user=user,
                            password=password,
                            host=host,
                            port=port,
                            database=database,
                            sslmode=sslmode)
    cursor = conn.cursor()

Answered By: Marc Dubois

psycopg2.InterfaceError: connection already closed / pgr_astar

Question:

Answers: