Pagination using python dictionaries

Question

What is the ideal and most performant way to paginate from dictionary with list of dicts?
For example if below is my dictionary:

items = {
    "device-1": [
        {"a": 1, "b": 15},
        {"a": 11, "b": 25},
        {"a": 21, "b": 35},
        {"a": 31, "b": 45},
        ]
    "device-2": [
        {"a": 100, "b": 150},
        {"a": 110, "b": 250},
        {"a": 210, "b": 350},
        {"a": 310, "b": 450},
        ]
}

If Page:1 and PageSize:3, the response should be:

items = {
    "device-1": [
        {"a": 1, "b": 15},
        {"a": 11, "b": 25},
        {"a": 21, "b": 35},
    ]
}

If Page:2 and PageSize:3, the response should be like:

items = {
    "device-1": [
        {"a": 31, "b": 45},
        ]
    "device-2": [
        {"a": 100, "b": 150},
        {"a": 110, "b": 250},
        ]
}

Asked By: N Raghu

||

Source

Answer 1

"Ideal" and "most performant" will depend on your particular application, but a relatively straightforward way to paginate without fully unpacking everything at once would be to create a flat stream of (device_key, item) tuples and then reassemble them:

from itertools import islice

def get_page(items, page_num, page_size):
    # Flatten into a stream of tuples (("device", item1), ("device", item2), ...)
    flat_iter = ((k, i) for (k, v) in items.items() for i in v)

    # Reassemble the tuples on the given page_num into a dict of lists
    page = {}
    for (k, i) in islice(flat_iter, (page_num-1)*page_size, page_num*page_size):
        page.setdefault(k, []).append(i)

    return page

Possibly you could squeak out something more performant by fast-forwarding through earlier devices, but the code is likely to be a lot bulkier.

Answered By: motto

Pagination using python dictionaries

Question:

Answers: