AWS CLI Can List S3 Bucket But Access Denied For Python Lambda

Question

I’ve used terraform to setup infra for an s3 bucket and my containerised lambda. I want to trigger the lambda to list the items in my s3 bucket. When I run the aws cli it’s fine:

aws s3 ls

returns

2022-11-08 23:04:19 bucket-name

This is my lambda:

import logging
import boto3

LOGGER = logging.getLogger(__name__)
LOGGER.setLevel(logging.DEBUG)

s3 = boto3.resource('s3')

def lambda_handler(event, context):
    LOGGER.info('Executing function...')
    bucket = s3.Bucket('bucket-name')
    total_objects = 0
    for i in bucket.objects.all():
        total_objects = total_objects + 1
    return {'total_objects': total_objects}

When I run the test in the AWS console, I’m getting this:

[ERROR] ClientError: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied

No idea why this is happening. These are my terraform lambda policies, roles and the s3 setup:

resource "aws_s3_bucket" "statements_bucket" {
  bucket = "bucket-name"
  acl    = "private"
}

resource "aws_s3_object" "object" {
  bucket = aws_s3_bucket.statements_bucket.id
  key    = "excel/"
}

resource "aws_iam_role" "lambda" {
  name               = "${local.prefix}-lambda-role"
  path               = "/"
  assume_role_policy = <<EOF
{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Action": "sts:AssumeRole",
           "Principal": {
               "Service": "lambda.amazonaws.com"
           },
           "Effect": "Allow"
       }
   ]
}
 EOF
}

resource "aws_iam_policy" "lambda" {
  name        = "${local.prefix}-lambda-policy"
  description = "S3 specified access"
  policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-name"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-name/*"
            ]
        }
    ]
}
    EOF
}

Asked By: clattenburg cake

||

Source

Answer 1

In order to run aws s3 ls, you would need to authorize the action s3:ListAllMyBuckets. This is because aws s3 ls lists all of your buckets.

You should be able to list the contents of your bucket using aws s3 ls s3://bucket-name. ~~However, you’re probably going to have to add "arn:aws:s3:::bucket-name/*" to the resource list for your role as well.~~ Edit: nevermind!

Answered By: dawaltco

Answer 2

As far as I can tell, your Lambda function has the correct IAM role (the one indicated in your Terraform template) but that IAM role has no attached policies.

You need to attach the S3 policy, and any other IAM policies needed, to the IAM role. For example:

resource "aws_iam_role_policy_attachment" "lambda-attach" {
  role       = aws_iam_role.role.name
  policy_arn = aws_iam_policy.policy.arn
}

Answered By: jarmod

AWS CLI Can List S3 Bucket But Access Denied For Python Lambda

Question:

Answers: