How to get the row count of a table instantly in DynamoDB?
Question:
I’m using boto.dynamodb2, and it seems I can use Table.query_count(). However it had raised an exception when no query filter is applied.
What can I do to fix this?
BTW, where is the document of filters that boto.dynamodb2.table.Table.Query can use? I tried searching for it but found nothing.
Answers:
There are two ways you can get a row count in DynamoDB.
The first is performing a full table scan and counting the rows as you go. For a table of any reasonable size this is generally a horrible idea as it will consume all of your provisioned read throughput.
The other way is to use the Describe Table request to get an estimate of the number of rows in the table. This will return instantly, but will only be updated periodically per the AWS documentation.
The number of items in the specified index. DynamoDB updates this
value approximately every six hours. Recent changes might not be
reflected in this value.
You can use this, to get count of entire table items
from boto.dynamodb2.table import Table
dynamodb_table = Table('Users')
dynamodb_table.count() # updated roughly 6 hours
Refer here: http://boto.cloudhackers.com/en/latest/ref/dynamodb2.html#module-boto.dynamodb2.table
query_count method will return the item count based on the indexes you provide.
For example,
from boto.dynamodb2.table import Table
dynamodb_table = Table('Users')
print dynamodb_table.query_count(
index='first_name-last_name-index', # Get indexes from indexes tab in dynamodb console
first_name__eq='John', # add __eq to your index name for specific search
last_name__eq='Smith' # This is your range key
)
You can add the primary index or global secondary indexes along with range keys.
possible comparison operators
__eq for equal
__lt for less than
__gt for greater than
__gte for greater than or equal
__lte for less than or equal
__between for between
__beginswith for begins with
Example for between
print dynamodb_table.query_count(
index='first_name-last_name-index', # Get indexes from indexes tab in dynamodb console
first_name__eq='John', # add __eq to your index name for specific search
age__between=[30, 50] # This is your range key
)
As per documentation boto3
“The number of items in the specified table. DynamoDB updates this value approximately every six hours. Recent changes might not be reflected in this value.”
import boto3
dynamoDBResource = boto3.resource('dynamodb')
table = dynamoDBResource.Table('tableName')
print(table.item_count)
or you can use DescribeTable:
import boto3
dynamoDBClient = boto3.client('dynamodb')
table = dynamoDBClient.describe_table(
TableName='tableName'
)
print(table)
If you want to count the number of items:
import boto3
client = boto3.client('dynamodb','us-east-1')
response = client.describe_table(TableName='test')
print(response['Table']['ItemCount'])
#ItemCount (integer) --The number of items in the specified table.
# DynamoDB updates this value approximately every six hours.
# Recent changes might not be reflected in this value.
Ref: Boto3 Documentation (under ItemCount in describe_table())
I’m using boto.dynamodb2, and it seems I can use Table.query_count(). However it had raised an exception when no query filter is applied.
What can I do to fix this?
BTW, where is the document of filters that boto.dynamodb2.table.Table.Query can use? I tried searching for it but found nothing.
There are two ways you can get a row count in DynamoDB.
The first is performing a full table scan and counting the rows as you go. For a table of any reasonable size this is generally a horrible idea as it will consume all of your provisioned read throughput.
The other way is to use the Describe Table request to get an estimate of the number of rows in the table. This will return instantly, but will only be updated periodically per the AWS documentation.
The number of items in the specified index. DynamoDB updates this
value approximately every six hours. Recent changes might not be
reflected in this value.
You can use this, to get count of entire table items
from boto.dynamodb2.table import Table
dynamodb_table = Table('Users')
dynamodb_table.count() # updated roughly 6 hours
Refer here: http://boto.cloudhackers.com/en/latest/ref/dynamodb2.html#module-boto.dynamodb2.table
query_count method will return the item count based on the indexes you provide.
For example,
from boto.dynamodb2.table import Table
dynamodb_table = Table('Users')
print dynamodb_table.query_count(
index='first_name-last_name-index', # Get indexes from indexes tab in dynamodb console
first_name__eq='John', # add __eq to your index name for specific search
last_name__eq='Smith' # This is your range key
)
You can add the primary index or global secondary indexes along with range keys.
possible comparison operators
__eq for equal
__lt for less than
__gt for greater than
__gte for greater than or equal
__lte for less than or equal
__between for between
__beginswith for begins with
Example for between
print dynamodb_table.query_count(
index='first_name-last_name-index', # Get indexes from indexes tab in dynamodb console
first_name__eq='John', # add __eq to your index name for specific search
age__between=[30, 50] # This is your range key
)
As per documentation boto3
“The number of items in the specified table. DynamoDB updates this value approximately every six hours. Recent changes might not be reflected in this value.”
import boto3
dynamoDBResource = boto3.resource('dynamodb')
table = dynamoDBResource.Table('tableName')
print(table.item_count)
or you can use DescribeTable:
import boto3
dynamoDBClient = boto3.client('dynamodb')
table = dynamoDBClient.describe_table(
TableName='tableName'
)
print(table)
If you want to count the number of items:
import boto3
client = boto3.client('dynamodb','us-east-1')
response = client.describe_table(TableName='test')
print(response['Table']['ItemCount'])
#ItemCount (integer) --The number of items in the specified table.
# DynamoDB updates this value approximately every six hours.
# Recent changes might not be reflected in this value.
Ref: Boto3 Documentation (under ItemCount in describe_table())