List database tables with SQLAlchemy

Question

I want to implement a function that gives information about all the tables (and their column names) that are present in a database (not only those created with SQLAlchemy). While reading the documentation it seems to me that this is done via reflection but I didn’t manage to get something working. Any suggestions or examples on how to do this?

Asked By: drgxfs

||

Source

Answer 1

Hey I created a small module that helps easily reflecting all tables in a database you connect to with SQLAlchemy, give it a look: EZAlchemy

from EZAlchemy.ezalchemy import EZAlchemy

DB = EZAlchemy(
    db_user='username',
    db_password='pezzword',
    db_hostname='127.0.0.1',
    db_database='mydatabase',
    d_n_d='mysql'   # stands for dialect+driver
)

# this function loads all tables in the database to the class instance DB
DB.connect()

# List all associations to DB, you will see all the tables in that database
dir(DB)

Answered By: PepperoniPizza

Answer 2

start with an engine:

from sqlalchemy import create_engine
engine = create_engine("postgresql://u:p@host/database")

quick path to all table /column names, use an inspector:

from sqlalchemy import inspect
inspector = inspect(engine)

for table_name in inspector.get_table_names():
   for column in inspector.get_columns(table_name):
       print("Column: %s" % column['name'])

docs: http://docs.sqlalchemy.org/en/rel_0_9/core/reflection.html?highlight=inspector#fine-grained-reflection-with-inspector

alternatively, use MetaData / Tables:

from sqlalchemy import MetaData
m = MetaData()
m.reflect(engine)
for table in m.tables.values():
    print(table.name)
    for column in table.c:
        print(column.name)

docs: http://docs.sqlalchemy.org/en/rel_0_9/core/reflection.html#reflecting-all-tables-at-once

Answered By: zzzeek

Answer 3

While reflection/inspection is useful, I had trouble getting the data out of the database. I found sqlsoup to be much more user-friendly. You create the engine using sqlalchemy and pass that engine to sqlsoup.SQlSoup. ie:

import sqlsoup

def create_engine():
    from sqlalchemy import create_engine
    return create_engine(f"mysql+mysqlconnector://{database_username}:{database_pw}@{database_host}/{database_name}")

def test_sqlsoup():
    engine = create_engine()
    db = sqlsoup.SQLSoup(engine)
    # Note: database must have a table called 'users' for this example
    users = db.users.all()
    print(users)

if __name__ == "__main__":
    test_sqlsoup()

If you’re familiar with sqlalchemy then you’re familiar with sqlsoup. I’ve used this to extract data from a wordpress database.

Answered By: user2682863

Answer 4

First set up the sqlalchemy engine.

from sqlalchemy import create_engine, inspect, text
from sqlalchemy.engine import url

connect_url = url.URL(
    'oracle',
    username='db_username',
    password='db_password',
    host='db_host',
    port='db_port',
    query=dict(service_name='db_service_name'))

engine = create_engine(connect_url)

try:
    engine.connect()
except Exception as error:
    print(error)
    return

Like others have mentioned, you can use the inspect method to get the table names.

But in my case, the list of tables returned by the inspect method was incomplete.

So, I found out another way to find table names by using pure SQL queries in sqlalchemy.

query = text("SELECT table_name FROM all_tables where owner = '%s'"%str('db_username'))

table_name_data = self.session.execute(query).fetchall()

Just for sake of completeness of answer, here’s the code to fetch table names by inspect method (if it works good in your case).

inspector = inspect(engine)
table_names = inspector.get_table_names()

Answered By: Rohit

Answer 5

I’m proposing another solution as I was not satisfied by any of the previous in the case of postgres which uses schemas. I hacked this solution together by looking into the pandas source code.

from sqlalchemy import MetaData, create_engine
from typing import List

def list_tables(pg_uri: str, schema: str) -> List[str]:
    with create_engine(pg_uri).connect() as conn:
        meta = MetaData(conn, schema=schema)
        meta.reflect(views=True)
        return list(meta.tables.keys())

In order to get a list of all tables in your schema, you need to form your postgres database uri pg_uri (e.g. "postgresql://u:p@host/database" as in the zzzeek’s answer) as well as the schema’s name schema. So if we use the example uri as well as the typical schema public we would get all the tables and views with:

list_tables("postgresql://u:p@host/database", "public")

Answered By: Zephaniah Grunschlag

List database tables with SQLAlchemy

Question:

Answers: