Failing to pre-sign s3 url in Bahrain AWS region ONLY

Question

I’ve had some Python code that pre-signs AWS S3 URLs that have been working for years. We just added a new bucket in the Bahrain AWS data center. This location was disabled and required explicitly enabling that data center.

That all seemed fine. However, the resulting URL always gives me an IllegalLocationConstraintException and I believe boto3 is generating the wrong domain name in the URL.

The error indicates that the request is sent to us-east-1. I know you have to specify the region, we have 8-10 regions we are using and they all are fine until this one.

The URL generated by boto3.client('s3', ...).generate_presigned_url() seems to give a URL for the us east S3. Given something like this:

region: me-south-1
bucket: bucket-name
key: full/path/to/file.txt

You would expect something like this for the signed URL:

https://bucket-name.s3.me-south-1.amazonaws.com/full/path/to/file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AZIAJ4252S33LNN3Y14Q/20100504/me-south-1/s3/aws4_request&X-Amz-Date=20200504TA22522Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=1717275aff4af5fcff2f44f74615fadb5ba448eb83219c88f59d0792d4e44b8f

Notice the domain: bucket-name.s3.me-south-1.amazonaws.com

Yet, what we get from boto3.client('s3', ...).generate_presigned_url() is:

https://bucket-name.s3.amazonaws.com/full/path/to/file.txt?X-Amz-Algorithm=...

Notice that this resulting domain is: s3.amazonaws.com

Has anyone had issues with the newer s3 locations like this?

If it helps, the full error message we get back is:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
   <Code>IllegalLocationConstraintException</Code>
   <Message>
   The me-south-1 location constraint is incompatible for the region specific endpoint this request was sent to.
   </Message>
</Error>

Asked By: Michael Kennedy

||

Source

Answer 1

Try specifying endpoint_url in S3 client:

boto3.client('s3', endpoint_url='https://s3.me-south-1.amazonaws.com', region_name='me-south-1')

If you get the following error

The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

specify signature_version too:

from botocore.client import Config
boto3.client('s3', config=Config(signature_version='s3v4'), endpoint_url='https://s3.me-south-1.amazonaws.com', region_name='me-south-1')

I have tried and it works for me ◡̈

Answered By: jellycsc

Answer 2

I have the same issue in af-south-1 and this is workaround, which works for me:

s3 = boto3.client('s3', region_name='af-south-1')
endpointUrl = s3.meta.endpoint_url
s3 = boto3.client('s3', endpoint_url=endpointUrl, region_name='af-south-1')

Yes, as you can see, boto3 S3 client object knows the proper URL but does not use it.

Of course it will be better to fix it in boto3.

Answered By: eNca

Failing to pre-sign s3 url in Bahrain AWS region ONLY

Question:

Answers: