Exception closing connection using sqlalchemy with asyncio and postgresql

Question

I have an API server using Python 3.7.10. I am using the FastAPI framework with sqlalchemy, asyncio, psycopg2-binary, asyncpg along with postgresql. I am deploying this using aws elasticbeanstalk. The application seems to work fine but everytime my frontend calls an endpoint, it seems like the connection is not closing correctly.

Error

Jun  1 21:17:33 web: ERROR:sqlalchemy.pool.impl.AsyncAdaptedQueuePool:Exception closing connection <AdaptedConnection <asyncpg.connection.Connection object at 0x7fd8b005cb90>>
Jun  1 21:17:33 web: Traceback (most recent call last):
Jun  1 21:17:33 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.7/site-packages/sqlalchemy/pool/base.py", line 247, in _close_connection
Jun  1 21:17:33 web: self._dialect.do_close(connection)
Jun  1 21:17:33 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.7/site-packages/sqlalchemy/engine/default.py", line 688, in do_close
Jun  1 21:17:33 web: dbapi_connection.close()
Jun  1 21:17:33 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.7/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 749, in close
Jun  1 21:17:33 web: self.await_(self._connection.close())
Jun  1 21:17:33 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.7/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 68, in await_only
Jun  1 21:17:33 web: return current.driver.switch(awaitable)
Jun  1 21:17:33 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.7/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 121, in greenlet_spawn
Jun  1 21:17:33 web: value = await result
Jun  1 21:17:33 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.7/site-packages/asyncpg/connection.py", line 1334, in close
Jun  1 21:17:33 web: await self._protocol.close(timeout)
Jun  1 21:17:33 web: File "asyncpg/protocol/protocol.pyx", line 581, in close
Jun  1 21:17:33 web: concurrent.futures._base.CancelledError

Here is my setup for the engine and session:

from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

from app.model.base import CustomBase
from app.core.config import SQLALCHEMY_DATABASE_URI

engine = create_async_engine(SQLALCHEMY_DATABASE_URI)

SessionLocal = sessionmaker(
    autocommit=False,
    autoflush=False,
    class_=AsyncSession,
    bind=engine,
    expire_on_commit=False,
)

I am using FastAPI’s dependency injection to get the session with the following:

async def get_db() -> AsyncSession:
    async with SessionLocal() as session:
        yield session

This error only shows up in my deployment and not my local environment, and seems to only when using sqlalchemy asynchronously with asyncio. Thanks for the help!

Asked By: Guillermo

||

Source

Answer 1

generally I had similar issue when using:

@app.middleware("http")
async def add_process_time_header(request: fastapi.Request, call_next):

Disabling middleware helped. Still trying to figure this out 🙂

Answered By: Norbert

Answer 2

My problem was fixed by using NullPool class.

from sqlalchemy.pool import NullPool
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

from app.model.base import CustomBase
from app.core.config import SQLALCHEMY_DATABASE_URI

engine = create_async_engine(
    SQLALCHEMY_DATABASE_URI, pool_pre_ping=True, poolclass=NullPool
)

SessionLocal = sessionmaker(
    autocommit=False,
    autoflush=False,
    class_=AsyncSession,
    bind=engine,
    expire_on_commit=False,
)

Answered By: Guillermo

Exception closing connection using sqlalchemy with asyncio and postgresql

Question:

Answers: