How to append a value to list attribute on AWS DynamoDB?
Question:
I’m using DynamoDB as an K-V db (cause there’s not much data, I think that’s fine) , and part of ‘V’ is list type (about 10 elements). There’s some session to append a new value to it, and I cannot find a way to do this in 1 request. What I did is like this:
item = self.list_table.get_item(**{'k': 'some_key'})
item['v'].append('some_value')
item.partial_save()
I request the server first and save it after modified the value. That’s not atomic and looks ugly. Is there any way to do this in one request?
Answers:
I would look at update expressions:
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.Modifying.html#Expressions.Modifying.UpdateExpressions.ADD
Should be doable with an ADD, although not sure what the support in boto is for this.
You can do this in 1 request by using the UpdateItem
API in conjunction with an UpdateExpression
. Since you want to append to a list, you would use the SET
action with the list_append
function:
SET
supports the following functions:
…
list_append (operand, operand)
– evaluates to a list with a new
element added to it. You can append the new element to the start or
the end of the list by reversing the order of the operands.
You can see a couple examples of this on the Modifying Items and Attributes with Update Expressions documentation:
-
The following example adds a new element to the FiveStar review list.
The expression attribute name #pr
is ProductReviews; the attribute
value :r
is a one-element list. If the list previously had two
elements, [0]
and [1]
, then the new element will be [2]
.
SET #pr.FiveStar = list_append(#pr.FiveStar, :r)
-
The following example adds another element to the FiveStar review
list, but this time the element will be appended to the start of the
list at [0]
. All of the other elements in the list will be shifted by
one.
SET #pr.FiveStar = list_append(:r, #pr.FiveStar)
The #pr
and :r
are using placeholders for the attribute names and values. You can see more information on those on the Using Placeholders for Attribute Names and Values documentation.
The following code should work with boto3:
table = get_dynamodb_resource().Table("table_name")
result = table.update_item(
Key={
'hash_key': hash_key,
'range_key': range_key
},
UpdateExpression="SET some_attr = list_append(some_attr, :i)",
ExpressionAttributeValues={
':i': [some_value],
},
ReturnValues="UPDATED_NEW"
)
if result['ResponseMetadata']['HTTPStatusCode'] == 200 and 'Attributes' in result:
return result['Attributes']['some_attr']
The get_dynamodb_resource method here is just:
def get_dynamodb_resource():
return boto3.resource(
'dynamodb',
region_name=os.environ['AWS_DYNAMO_REGION'],
endpoint_url=os.environ['AWS_DYNAMO_ENDPOINT'],
aws_secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'],
aws_access_key_id=os.environ['AWS_ACCESS_KEY_ID'])
@LaserJesus ‘s answer is correct. However, using boto3 directly is kind of a pain, hard to maintain, and not at all reusable. dynamof abstracts that junk away. Using dynamof
appending an item to a list attribute would look like:
from functools import partial
from boto3 import client
from dynamof.executor import execute
from dynamof.operations import update
from dynamof.attribute import attr
client = client('dynamodb', endpoint_url='http://localstack:4569')
db = partial(execute, client)
db(update(
table_name='users',
key={ 'id': user_id },
attributes={
'roles': attr.append('admin')
}))
disclaimer: I wrote dynamof
I’m using DynamoDB as an K-V db (cause there’s not much data, I think that’s fine) , and part of ‘V’ is list type (about 10 elements). There’s some session to append a new value to it, and I cannot find a way to do this in 1 request. What I did is like this:
item = self.list_table.get_item(**{'k': 'some_key'})
item['v'].append('some_value')
item.partial_save()
I request the server first and save it after modified the value. That’s not atomic and looks ugly. Is there any way to do this in one request?
I would look at update expressions:
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.Modifying.html#Expressions.Modifying.UpdateExpressions.ADD
Should be doable with an ADD, although not sure what the support in boto is for this.
You can do this in 1 request by using the UpdateItem
API in conjunction with an UpdateExpression
. Since you want to append to a list, you would use the SET
action with the list_append
function:
SET
supports the following functions:…
list_append (operand, operand)
– evaluates to a list with a new
element added to it. You can append the new element to the start or
the end of the list by reversing the order of the operands.
You can see a couple examples of this on the Modifying Items and Attributes with Update Expressions documentation:
The following example adds a new element to the FiveStar review list.
The expression attribute name#pr
is ProductReviews; the attribute
value:r
is a one-element list. If the list previously had two
elements,[0]
and[1]
, then the new element will be[2]
.SET #pr.FiveStar = list_append(#pr.FiveStar, :r)
The following example adds another element to the FiveStar review
list, but this time the element will be appended to the start of the
list at[0]
. All of the other elements in the list will be shifted by
one.SET #pr.FiveStar = list_append(:r, #pr.FiveStar)
The #pr
and :r
are using placeholders for the attribute names and values. You can see more information on those on the Using Placeholders for Attribute Names and Values documentation.
The following code should work with boto3:
table = get_dynamodb_resource().Table("table_name")
result = table.update_item(
Key={
'hash_key': hash_key,
'range_key': range_key
},
UpdateExpression="SET some_attr = list_append(some_attr, :i)",
ExpressionAttributeValues={
':i': [some_value],
},
ReturnValues="UPDATED_NEW"
)
if result['ResponseMetadata']['HTTPStatusCode'] == 200 and 'Attributes' in result:
return result['Attributes']['some_attr']
The get_dynamodb_resource method here is just:
def get_dynamodb_resource():
return boto3.resource(
'dynamodb',
region_name=os.environ['AWS_DYNAMO_REGION'],
endpoint_url=os.environ['AWS_DYNAMO_ENDPOINT'],
aws_secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'],
aws_access_key_id=os.environ['AWS_ACCESS_KEY_ID'])
@LaserJesus ‘s answer is correct. However, using boto3 directly is kind of a pain, hard to maintain, and not at all reusable. dynamof abstracts that junk away. Using dynamof
appending an item to a list attribute would look like:
from functools import partial
from boto3 import client
from dynamof.executor import execute
from dynamof.operations import update
from dynamof.attribute import attr
client = client('dynamodb', endpoint_url='http://localstack:4569')
db = partial(execute, client)
db(update(
table_name='users',
key={ 'id': user_id },
attributes={
'roles': attr.append('admin')
}))
disclaimer: I wrote dynamof