SQLAlchemy min/max of query with limit

Question

Hello everyone,

I have a couple of sensors reading temperature and humidity. With SQLAlchemy I succeeded to query the last ten results of every sensor with:

 sensorlist = db.session.query(Sensor).order_by(Sensor.id.asc()).all()
    for sensor in sensorlist:
        readings = sensor.readings.order_by(Sensorreading.time.desc()).limit(10)

Now I want of the ten results the min and the max, so I can present that of every sensor. Something like:

<sensor 1, min temp = 23, max temp = 26, min hum = 56, max hum =74>
<sensor 2, min temp = 22, max temp = 30, min hum = 51, max hum =63>
<sensor 3, min temp = 24, max temp = 25, min hum = 62, max hum =75>

But I don’t know how to query that with SQLAlchemy/python. For every reading I did it with this code, but then I can’t get it work to limit the data before calculate min/max. If I replace .all() for .limit(10), then it just query ten sensors, instead of ten readings:

sensors = db.session.query(Sensor.name, db.func.min(Sensorreading.temp_value).label("temp_value_min"), db.func.max(Sensorreading.temp_value).label("temp_value_max")).outerjoin(Sensorreading, Sensor.id == Sensorreading.sensor_id).group_by(Sensor.name).all()

Just for some background information the models of my table:

class Sensor(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(50))
    sensor_type = db.Column(db.Integer)
    pin = db.Column(db.Integer)
    limit_temp_up = db.Column(db.Float)
    limit_temp_down = db.Column(db.Float)
    limit_hum_up = db.Column(db.Float)
    limit_hum_down = db.Column(db.Float)
    limit_aqua_temp_up = db.Column(db.Float)
    limit_aqua_temp_down = db.Column(db.Float)
    readings = db.relationship('Sensorreading', backref='sensor', lazy='dynamic')

class Sensorreading(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    time = db.Column(db.DateTime)
    name = db.Column(db.String(50))
    temp_value = db.Column(db.Float)
    hum_value = db.Column(db.Float)
    aqua_temp_value = db.Column(db.Float)
    sensor_id = db.Column(db.Integer, db.ForeignKey('sensor.id'))

Hope someone can help. Any tip, tricks, code helps or links are welcome! Thanks in advance.

Asked By: Thijs

||

Source

Answer 1

It will return the entire list and you can use slicing in python list

sensors = db.session.query(Sensor.name, db.func.min(Sensorreading.temp_value).label("temp_value_min"), db.func.max(Sensorreading.temp_value).label("temp_value_max")).outerjoin(Sensorreading, Sensor.id == Sensorreading.sensor_id).group_by(Sensor.name).all()[:10]

Answered By: mad_

Answer 2

I think is better to use limit and offset in this case. If you return a big data set from your sql and slice the set in memory that would cause some unnecessary overhead.

This solution will return a data set with 10 items(the pagination happens in the sql side):

sensors = db.session.query(Sensor.name, db.func.min(Sensorreading.temp_value).label("temp_value_min"), db.func.max(Sensorreading.temp_value).label("temp_value_max")).outerjoin(Sensorreading, Sensor.id == Sensorreading.sensor_id).group_by(Sensor.name).limit(10).offset(0).all()

For offset you need the first item – 1. So page 1 start with item 1 you need the offset as 0. For page 2 offset will be 11 – 1 = 10.

Answered By: Flavio Garcia

Answer 3

You can do this on the database side by first creating a common table expression that ranks each row for each sensor by id, and then group by the sensor but only over the first ten rows for each sensor.

with Session() as s:
    # Assign each row a number based on sensor and id.    
    cte = sa.select(
        Sensorreading.sensor_id,
        Sensorreading.temp_value,
        sa.func.row_number()
        .over(partition_by=Sensorreading.sensor_id, order_by=Sensorreading.id)
        .label('rn'),
    ).cte('cte')
    
    # Now take the max grouped by sensor name, but only for 
    # the first ten rows for each sensor.
    q = ( 
        sa.select(Sensor.name, sa.func.max(cte.c.temp_value))
        .join(cte, Sensor.id == cte.c.sensor_id)
        .where(cte.c.rn <= 10)
        .group_by(Sensor.name)
        .order_by(Sensor.name)
    )
 
    res = s.execute(q)
    for row in res:
        print(row)

This example uses standard SQLAlchemy; it can be mostly converted to Flask-SQLAlchemy by removing the with line, changing s to db.session and converting sa. to db..

Answered By: snakecharmerb

SQLAlchemy min/max of query with limit

Question:

Answers: