How to create the dynamodb table using serverless.yml and delete the items of it using python boto3?
Question:
I’ve created the dynamodb table using serverless.yml as below:
resources:
Resources:
myTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: myTable
AttributeDefinitions:
- AttributeName: id
AttributeType: S
- AttributeName: firstname
AttributeType: S
- AttributeName: lastname
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: firstname
KeyType: RANGE
BillingMode: PAY_PER_REQUEST
SSESpecification:
SSEEnabled: true
But I’ve got this issue:
An error occurred: myTable – One or more parameter values were
invalid: Number of attributes in KeySchema does not exactly match
number of attributes defined in AttributeDefinitions (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException;
Request ID: PEI9OT7E72HQN4N5MQUOIUQ18JVV4KQNSO5AEMVJF66Q9ASUAAJG;
Proxy: null).
Could you help me creating the dynamodb table using serverless.yml?
And how can I delete the items that first name is "First" in this table using python boto3?
Answers:
Reason is that all of your AttributeNames in AttributeDefinitions
must be included in the KeySchema
as well. I can see the lastname
attribute is missing in there.
If you want to keep your KeySchema
you have to drop lastname
from AttributeDefinitions
:
Resources:
myTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: myTable
AttributeDefinitions:
- AttributeName: id
AttributeType: S
- AttributeName: firstname
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: firstname
KeyType: RANGE
BillingMode: PAY_PER_REQUEST
SSESpecification:
SSEEnabled: true
But if you want to keep lastname
, you could define local secondary index for your table:
Resources:
myTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: myTable
AttributeDefinitions:
- AttributeName: id
AttributeType: S
- AttributeName: firstname
AttributeType: S
- AttributeName: lastname
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: firstname
KeyType: RANGE
LocalSecondaryIndexes:
- IndexName: by-lastname
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: lastname
KeyType: RANGE
Projection:
ProjectionType: ALL
BillingMode: PAY_PER_REQUEST
SSESpecification:
SSEEnabled: true
With the above, you can sort for lastname
using the LSI, while firstname
would be used in the primary table.
how can I delete the items that first name is "First" in this table using python boto3?
You can’t do it directly with or without boto3, unless you want to perform scan, which should be avoided as it can be expensive and is not efficient. To general solution is to define a Global Secondary Indexes where the firstname
would be the new primary key. Then you would query the GSI for the firstname
of interest to obtain the id
of the record you want to delete. If you have several records with same firstname
, which probably will be the case, you get a number of records back (GSI primary keys don’t need to be unique, unlike for the primary table).
Example table with LSI and GSI:
Resources:
myTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: myTable
AttributeDefinitions:
- AttributeName: id
AttributeType: S
- AttributeName: firstname
AttributeType: S
- AttributeName: lastname
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: firstname
KeyType: RANGE
LocalSecondaryIndexes:
- IndexName: by-lastname
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: lastname
KeyType: RANGE
Projection:
ProjectionType: ALL
GlobalSecondaryIndexes:
- IndexName: firstname-gsi
KeySchema:
- AttributeName: firstname
KeyType: HASH
Projection:
ProjectionType: ALL
#ProvisionedThroughput:
# ProvisionedThroughput
BillingMode: PAY_PER_REQUEST
SSESpecification:
SSEEnabled: true
You just need to remove the ‘lastname’ attribute from the AttributeDefinitions. Anything defined as an AttributeDefinitions must be used in the KeySchema section. Don’t forget to export your TableName, Arn, and StreamArn so that you can reference them in other services and IAM policies.
https://carova.io/snippets/serverless-aws-cloudformation-output-stack-variables
Another bonus for exporting variables like the TableName or StreamArn in your CloudFormation Stack is you can reference them across different CloudFormation Stacks.
https://carova.io/snippets/serverless-aws-reference-other-cloudformation-stack-variables
I’ve created the dynamodb table using serverless.yml as below:
resources:
Resources:
myTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: myTable
AttributeDefinitions:
- AttributeName: id
AttributeType: S
- AttributeName: firstname
AttributeType: S
- AttributeName: lastname
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: firstname
KeyType: RANGE
BillingMode: PAY_PER_REQUEST
SSESpecification:
SSEEnabled: true
But I’ve got this issue:
An error occurred: myTable – One or more parameter values were
invalid: Number of attributes in KeySchema does not exactly match
number of attributes defined in AttributeDefinitions (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException;
Request ID: PEI9OT7E72HQN4N5MQUOIUQ18JVV4KQNSO5AEMVJF66Q9ASUAAJG;
Proxy: null).
Could you help me creating the dynamodb table using serverless.yml?
And how can I delete the items that first name is "First" in this table using python boto3?
Reason is that all of your AttributeNames in AttributeDefinitions
must be included in the KeySchema
as well. I can see the lastname
attribute is missing in there.
If you want to keep your KeySchema
you have to drop lastname
from AttributeDefinitions
:
Resources:
myTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: myTable
AttributeDefinitions:
- AttributeName: id
AttributeType: S
- AttributeName: firstname
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: firstname
KeyType: RANGE
BillingMode: PAY_PER_REQUEST
SSESpecification:
SSEEnabled: true
But if you want to keep lastname
, you could define local secondary index for your table:
Resources:
myTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: myTable
AttributeDefinitions:
- AttributeName: id
AttributeType: S
- AttributeName: firstname
AttributeType: S
- AttributeName: lastname
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: firstname
KeyType: RANGE
LocalSecondaryIndexes:
- IndexName: by-lastname
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: lastname
KeyType: RANGE
Projection:
ProjectionType: ALL
BillingMode: PAY_PER_REQUEST
SSESpecification:
SSEEnabled: true
With the above, you can sort for lastname
using the LSI, while firstname
would be used in the primary table.
how can I delete the items that first name is "First" in this table using python boto3?
You can’t do it directly with or without boto3, unless you want to perform scan, which should be avoided as it can be expensive and is not efficient. To general solution is to define a Global Secondary Indexes where the firstname
would be the new primary key. Then you would query the GSI for the firstname
of interest to obtain the id
of the record you want to delete. If you have several records with same firstname
, which probably will be the case, you get a number of records back (GSI primary keys don’t need to be unique, unlike for the primary table).
Example table with LSI and GSI:
Resources:
myTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: myTable
AttributeDefinitions:
- AttributeName: id
AttributeType: S
- AttributeName: firstname
AttributeType: S
- AttributeName: lastname
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: firstname
KeyType: RANGE
LocalSecondaryIndexes:
- IndexName: by-lastname
KeySchema:
- AttributeName: id
KeyType: HASH
- AttributeName: lastname
KeyType: RANGE
Projection:
ProjectionType: ALL
GlobalSecondaryIndexes:
- IndexName: firstname-gsi
KeySchema:
- AttributeName: firstname
KeyType: HASH
Projection:
ProjectionType: ALL
#ProvisionedThroughput:
# ProvisionedThroughput
BillingMode: PAY_PER_REQUEST
SSESpecification:
SSEEnabled: true
You just need to remove the ‘lastname’ attribute from the AttributeDefinitions. Anything defined as an AttributeDefinitions must be used in the KeySchema section. Don’t forget to export your TableName, Arn, and StreamArn so that you can reference them in other services and IAM policies.
https://carova.io/snippets/serverless-aws-cloudformation-output-stack-variables
Another bonus for exporting variables like the TableName or StreamArn in your CloudFormation Stack is you can reference them across different CloudFormation Stacks.
https://carova.io/snippets/serverless-aws-reference-other-cloudformation-stack-variables