How can I easily determine if a Boto 3 S3 bucket resource exists?

Question:

For example, I have this code:

import boto3

s3 = boto3.resource('s3')

bucket = s3.Bucket('my-bucket-name')

# Does it exist???
Asked By: Daniel

||
Source

Answers:

At the time of this writing there is no high-level way to quickly check whether a bucket exists and you have access to it, but you can make a low-level call to the HeadBucket operation. This is the most inexpensive way to do this check:

from botocore.client import ClientError

try:
    s3.meta.client.head_bucket(Bucket=bucket.name)
except ClientError:
    # The bucket does not exist or you have no access.

Alternatively, you can also call create_bucket repeatedly. The operation is idempotent, so it will either create or just return the existing bucket, which is useful if you are checking existence to know whether you should create the bucket:

bucket = s3.create_bucket(Bucket='my-bucket-name')

As always, be sure to check out the official documentation.

Note: Before the 0.0.7 release, meta was a Python dictionary.

Answered By: Daniel

As mentioned by @Daniel, the best way as suggested by Boto3 docs is to use head_bucket()

head_bucket() – This operation is useful to determine if a
bucket exists and you have permission to access it.

If you have a small number of buckets, you can use the following:

>>> import boto3
>>> s3 = boto3.resource('s3')
>>> s3.Bucket('Hello') in s3.buckets.all()
False
>>> s3.Bucket('some-docs') in s3.buckets.all()
True
>>>
Answered By: helloV

you can use conn.get_bucket

from boto.s3.connection import S3Connection
from boto.exception import S3ResponseError    

conn = S3Connection(aws_access_key, aws_secret_key)

try:
    bucket = conn.get_bucket(unique_bucket_name, validate=True)
except S3ResponseError:
    bucket = conn.create_bucket(unique_bucket_name)

quoting the documentation at http://boto.readthedocs.org/en/latest/s3_tut.html

As of Boto v2.25.0, this now performs a HEAD request (less expensive but worse error messages).

Answered By: vim

Use lookup Function -> Returns None if bucket Exist 

if s3.lookup(bucketName) is None:
    bucket=s3.create_bucket(bucketName) # Bucket Don't Exist
else:
    bucket = s3.get_bucket(bucketName) #Bucket Exist
Answered By: Jacob Oommen

I tried Daniel’s example and it was really helpful. Followed up the boto3 documentation and here is my clean test code. I have added a check for ‘403’ error when buckets are private and return a ‘Forbidden!’ error.

import boto3, botocore
s3 = boto3.resource('s3')
bucket_name = 'some-private-bucket'
#bucket_name = 'bucket-to-check'

bucket = s3.Bucket(bucket_name)
def check_bucket(bucket):
    try:
        s3.meta.client.head_bucket(Bucket=bucket_name)
        print("Bucket Exists!")
        return True
    except botocore.exceptions.ClientError as e:
        # If a client error is thrown, then check that it was a 404 error.
        # If it was a 404 error, then the bucket does not exist.
        error_code = int(e.response['Error']['Code'])
        if error_code == 403:
            print("Private Bucket. Forbidden Access!")
            return True
        elif error_code == 404:
            print("Bucket Does Not Exist!")
            return False

check_bucket(bucket)

Hope this helps some new into boto3 like me.

Answered By: mondnom

I’ve had success with this:

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket-name')

if bucket.creation_date:
   print("The bucket exists")
else:
   print("The bucket does not exist")
Answered By: href_