How can I catch all errors from a create.sql just like SQLite gives?

Question

I have a create.sql (and a populate.sql) that create a SQLite3 database (and populate it with some dummy data). I then save the resulting database and make some further analysis.
I use Python to automate this process for several pairs of sql files.

db_mem = sqlite3.connect(":memory:")
cur = db_mem.cursor()
try:
    with open(row['path']+'/' + 'create.sql') as fp:
        cur.executescript(fp.read())  
except:
    creates.append('C-error')
    continue
else:
    creates.append('Ok')
    try:
        with open(row['path']+'/' + 'populate.sql') as fp:
            cur.executescript(fp.read())  
    except sqlite3.Error as x:
        populates.append('P-error')
    else:
        populates.append('Ok')
        db_disk = sqlite3.connect(f'./SQL_Final/{index}_out.db')
        db_mem.backup(db_disk)

However, i can only catch 1 error at creation, instead of several errors that are outputed when I do .read create.sql from sqlite.
My question is how can I catch all errors?

For reproducibility issues, here is a dummy create.sql (that generates errors):

DROP TABLE IF EXISTS Comp;
CREATE TABLE Comp (
    idComp INTEGER PRIMARY KEY,
    nomeComp TEXT,
    dataInicio TEXT,
    dataFim TEXT
);
DROP TABLE IF EXISTS Game;
DROP TABLE IF EXISTS Stade;
DROP TABLE IF EXISTS Club;
DROP TABLE IF EXISTS Squad;
CREATE TABLE Game (
    idGame INTEGER PRIMARY KEY,
    golosFora INTEGER,
    golosCasa INTEGER,
    data DATE,
    jornada INTEGER,
    duração TIME,
    nomeComp TEXT REFERENCES Comp,
    nomeStade TEXT REFERENCES Stade,
    nomeSquadFora TEXT REFERENCES Squad,
    nomeSquadCasa TEXT REFERENCES Squad,
    CONSTRAINT CHECK_Game_golosFora CHECK (Game_golosFora >= 0),
    CONSTRAINT CHECK_Game_golosCasa CHECK (Game_golosCasa >= 0),
    CONSTRAINT CHECK_Game_jornada CHECK (jornada > 0)
);
CREATE TABLE Stade (
    nomeStade TEXT PRIMARY KEY,
    local TEXT NOT NULL,
    idGame INTEGER REFERENCES Game
);
CREATE TABLE Club (
    nomeClub TEXT PRIMARY KEY,
    país TEXT NOT NULL
);
CREATE TABLE Squad (
    nomeSquad TEXT PRIMARY Key,
    nomeClub TEXT REFERENCES Club
);

If I read this file from SQLIte (with .read create.sql) I get the following errors:

Error: near line 2: in prepare, table "Comp" has more than one primary key (1)
Error: near line 13: in prepare, no such column: Game_golosFora (1)

However, if I automate from Python, I get only a single error:

Error: table "Comp" has more than one primary key

Is there any way I can fix this?

Asked By: MrT77

||

Source

Answer 1

The executescript() will run the whole sql file but will raise exception when an error occurs.

Since you want to continue the DB schema creation regardless of errors, you could switch from using executescript() to looping over the sql statements list by calling the execute() on each of them.

This example will demonstrate that the processing continues when an error occurs. We will try to create a table that already exists to simulate sqlite error.

create.sql:

CREATE TABLE Stade (
    nomeStade TEXT PRIMARY KEY,
    local TEXT NOT NULL,
    idGame INTEGER REFERENCES Game
);

CREATE TABLE Stade (
    nomeStade TEXT PRIMARY KEY,
    local TEXT NOT NULL,
    idGame INTEGER REFERENCES Game
);

CREATE TABLE Stade2 (
    nomeStade TEXT PRIMARY KEY,
    local TEXT NOT NULL,
    idGame INTEGER REFERENCES Game
);

CREATE TABLE Stade2 (
    nomeStade TEXT PRIMARY KEY,
    local TEXT NOT NULL,
    idGame INTEGER REFERENCES Game
);

CREATE TABLE Stade3 (
    nomeStade TEXT PRIMARY KEY,
    local TEXT NOT NULL,
    idGame INTEGER REFERENCES Game
);

python:

import traceback
import sqlite3
db_mem = sqlite3.connect(":memory:")
cur = db_mem.cursor()
all_errors = []
try:
    with open('create.sql', 'r') as fp:
        text = fp.read().split(';')
        for sql in text:
            try:
                cur.execute(sql)
            except:
                # print("exception at sql: {}, details: {}".format(sql, traceback.format_exc()))
                all_errors.append(traceback.format_exc())
except:
    print("exception, details: {}".format(traceback.format_exc()))
    
else:
    db_disk = sqlite3.connect('db.sqlite')
    db_mem.backup(db_disk)

for i, error in enumerate(all_errors):
    print("Error #{}: {}".format(i, error)

running it, we can see 2 error messages, and inspecting the db.sqlite we can see the 3rd table as well, i.e. the processing reached the last sql.

Answered By: ilias-sp

How can I catch all errors from a create.sql just like SQLite gives?

Question:

Answers: