Number of deleted messages is twice as many as received messages

Question

I created SQS service with terraform

resource "aws_sqs_queue" "ses_queue" {
  name                       = "ses_queue"
  message_retention_seconds  = 86400
  receive_wait_time_seconds  = 1
  visibility_timeout_seconds = 15
}

resource "aws_lambda_event_source_mapping" "send_email_message" {
  event_source_arn = aws_sqs_queue.ses_queue.arn
  function_name    = aws_lambda_function.send_email_message.function_name
  batch_size       = 5
}

I am sending emails using lambda function

for record in event.get("Records"):
    receipt_handle = record.get("receiptHandle", "")
    request_body = record.get("body")
    response = send_email(request_body)
    if response:
        sqs_client.delete_message(QueueUrl=constants.SES_QUEUE_URL, ReceiptHandle=receipt_handle)

I am wondering why number of deleted messages is twice as many as received messages

Asked By: psowa001

||

Source

Answer 1

The Lambda Event Source Mapping already deletes the messages from the Queue if your Lambda Function terminates without errors:

If your function successfully processes the batch, Lambda deletes the messages from the queue.
— Source

In your Function Code you explicitly delete the messages you processed as well, this means the delete happens twice.

Answered By: Maurice

Answer 2

Fixed it with reporting batch item failures

terraform

resource "aws_lambda_event_source_mapping" "send_email_message" {
  event_source_arn         = aws_sqs_queue.ses_queue.arn
  function_name            = aws_lambda_function.send_email_message.function_name
  batch_size               = 5
  function_response_types  = ["ReportBatchItemFailures"]
}

lambda

reprocess_messages = []
for record in event.get("Records"):
    receipt_handle = record.get("receiptHandle", "")
    message_id = record.get("messageId", "")
    request_body = record.get("body")
    response = send_email(request_body)
    if not response:
        reprocess_messages.append({"itemIdentifier": message_id})

return {"batchItemFailures": reprocess_messages}

Answered By: psowa001

Number of deleted messages is twice as many as received messages

Question:

Answers: