Azure Storage Table returning empty entities

Question

I have the following Python code:

def GetData(tableService, tableName, dataFilter):
    keyMarkers = {}
    keyMarkers['nextpartitionkey'] = 0
    keyMarkers['nextrowkey'] = 0
    b=[]
    while True:
        #get a batch of data
        a = tableService.query_entities(table_name=tableName, filter=dataFilter,num_results=1000 ,marker=keyMarkers)
        #copy results to list
        for item in a.items:
            b.append(item.amount.value)
        #check to see if more data is available
        if len(a.next_marker) == 0:
            del a
            break
        #if more data available setup current position
        keyMarkers['nextpartitionkey'] = a.next_marker['nextpartitionkey']
        keyMarkers['nextrowkey'] = a.next_marker['nextrowkey']
        #house keep temp storage
        del a
    #return final list
    return b

It is working, however, for the first results, a.items comes empty. After 10 queries, it finally starts returning some data. It is like the table has no rows for the values I queried. I know it has, and it eventually comes. But only after returning too many empty results.

I have a PHP code for the same purpose, but it doesn’t get empty entities. So I don’t think this is a common behavior from Azure. Maybe the way its SDK for Python works?

Asked By: rlcabral

||

Source

Answer 1

On the very first request, can you try passing in ‘None’ for the marker, rather than a dictionary with 0 and 0 for nextpk / nextrk? I’m not sure, but this might be confusing the service into searching for a table entity with this pk & rk.

Answered By: Adam Sorrin – MSFT

Answer 2

Although it’s very late since this question was asked and this library (azure-cosmosdb-table) seems to have been deprecated since then (see here), I will answer this because I just faced this issue as well while using this library and it might help others.

The issue with a.items coming up empty for the first few tries is that the query_entities is unable to find any new entities in the PHYSICAL PARTITION in which it is searching for items. Where as, once you use the marker information correctly, it keeps trying to find newer values on every request until the marker has reached the point where it actually does locate items as per your dataFilter value. One more thing you may note is that there is a possibility that you will sometimes get less or more values depending on the num_results param you set. This is due to the physical partition only having that many values left in that partition for the dataFilter you applied.

Answered By: Saif Ullah

Azure Storage Table returning empty entities

Question:

Answers: