How to specify credentials when connecting to boto3 S3?

Question

On boto I used to specify my credentials when connecting to S3 in such a way:

import boto
from boto.s3.connection import Key, S3Connection
S3 = S3Connection( settings.AWS_SERVER_PUBLIC_KEY, settings.AWS_SERVER_SECRET_KEY )

I could then use S3 to perform my operations (in my case deleting an object from a bucket).

With boto3 all the examples I found are such:

import boto3
S3 = boto3.resource( 's3' )
S3.Object( bucket_name, key_name ).delete()

I couldn’t specify my credentials and thus all attempts fail with InvalidAccessKeyId error.

How can I specify credentials with boto3?

Asked By: Robert Brax

||

Source

Answer 1

You can create a session:

import boto3
session = boto3.Session(
    aws_access_key_id=settings.AWS_SERVER_PUBLIC_KEY,
    aws_secret_access_key=settings.AWS_SERVER_SECRET_KEY,
)

Then use that session to get an S3 resource:

s3 = session.resource('s3')

Answered By: Alasdair

Answer 2

You can get a client with new session directly like below.

 s3_client = boto3.client('s3', 
                      aws_access_key_id=settings.AWS_SERVER_PUBLIC_KEY, 
                      aws_secret_access_key=settings.AWS_SERVER_SECRET_KEY, 
                      region_name=REGION_NAME
                      )

Answered By: Rajez

Answer 3

There are numerous ways to store credentials while still using boto3.resource().
I’m using the AWS CLI method myself. It works perfectly.

https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html?fbclid=IwAR2LlrS4O2gYH6xAF4QDVIH2Q2tzfF_VZ6loM3XfXsPAOR4qA-pX_qAILys

Answered By: JustAGuy

Answer 4

I’d like expand on @JustAGuy’s answer. The method I prefer is to use AWS CLI to create a config file. The reason is, with the config file, the CLI or the SDK will automatically look for credentials in the ~/.aws folder. And the good thing is that AWS CLI is written in python.

You can get cli from pypi if you don’t have it already. Here are the steps to get cli set up from terminal

$> pip install awscli  #can add user flag 
$> aws configure
AWS Access Key ID [****************ABCD]:[enter your key here]
AWS Secret Access Key [****************xyz]:[enter your secret key here]
Default region name [us-west-2]:[enter your region here]
Default output format [None]:

After this you can access boto and any of the api without having to specify keys (unless you want to use a different credentials).

Answered By: x85ms16

Answer 5

This is older but placing this here for my reference too. boto3.resource is just implementing the default Session, you can pass through boto3.resource session details.

Help on function resource in module boto3:

resource(*args, **kwargs)
    Create a resource service client by name using the default session.

    See :py:meth:`boto3.session.Session.resource`.

https://github.com/boto/boto3/blob/86392b5ca26da57ce6a776365a52d3cab8487d60/boto3/session.py#L265

you can see that it just takes the same arguments as Boto3.Session

import boto3
S3 = boto3.resource('s3', region_name='us-west-2', aws_access_key_id=settings.AWS_SERVER_PUBLIC_KEY, aws_secret_access_key=settings.AWS_SERVER_SECRET_KEY)
S3.Object( bucket_name, key_name ).delete()

Answered By: DataDecay

Answer 6

you can set default aws env variables for secret and access keys – that way you dont need to change default client creation code – though it is better to pass it as a parameter if you have non-default creds

Answered By: Den Roman

Answer 7

If you rely on your .aws/credentials to store id and key for a user, it will be picked up automatically.

For instance

session = boto3.Session(profile_name='dev')
s3 = session.resource('s3')

This will pick up the dev profile (user) if your credentials file contains the following:

[dev]
aws_access_key_id = AAABBBCCCDDDEEEFFFGG
aws_secret_access_key = FooFooFoo
region=op-southeast-2

Answered By: Bernard

How to specify credentials when connecting to boto3 S3?

Question:

Answers: