Python – Fine Uploader Server Side AWS Version 4 signing request

My answer will be assuming you are using python3. It is good for AWS v4 signing for file chunking or regular uploads. First, in your javascript, ensure you have version:4 specified in the signature section:

 <script type="text/javascript">
     // execute the code after the document is loaded
     document.addEventListener("DOMContentLoaded", function() {
         // The code
        (function() {
            var uploader = new qq.s3.FineUploader({
                debug: true,
                element: document.getElementById('fine-uploader'),
                cors: {
                  expected: true
                },
                objectProperties: {
                      bucket: '<your bucket>',
                      region: 'cn-north-1',
                      acl: 'private',
                      key: "uuid",
                    },
                request: {
                    endpoint: 'https://<your bucket>.s3.cn-north-1.amazonaws.com.cn',
                    accessKey: '<your public access key>',

                },
                signature: {
                    endpoint: '{{ url_for('data.s3_signature') }}',
                    version: 4,
                    customHeaders: {
                              "X-CSRF-Token": $("meta[name='csrf-token']").attr("content")
                            }
                },
                uploadSuccess: {
                    endpoint: '{{ url_for('data.s3_success') }}'
                },
                iframeSupport: {
                    localBlankPagePath: '/success.html'
                },
                chunking: {
                    enabled: true,
                    concurrent: {
                        enabled: true
                    }
                },
                resume: {
                    enabled: true
                },
                retry: {
                   enableAuto: true // defaults to false
                },
                deleteFile: {
                    enabled: true,
                    endpoint: '{{ url_for('data.s3_delete', key=key) }}'
                }
            })
        }());
    });
    </script>

Next, three helper methods for signing and one for grabbing a date from policy date data sent from fineuploader:

def hash_sha256(msg:str):
    """
    Generate a SHA256 hash and return the base16 Uicode string.

    msg -- A Unicode message to hash.

    """
    return binascii.hexlify(hashlib.sha256(
        bytearray(msg.strip(), 'utf-8')).digest()).decode('utf-8')

def sign_sha256(key, msg):
    """
    Generate an SHA256 HMAC, encoding msg to UTF-8 if not
    already encoded.

    key -- signing key. bytes.
    msg -- message to sign. unicode or bytes.

    """
    if isinstance(msg, text_type):
        msg = msg.encode('utf-8')
    return hmac.new(key, msg, hashlib.sha256).digest()

def generate_key(cls, secret_key, region, service, date,
                 intermediates=False):
    """
    Generate the signing key string as bytes.

    If intermediate is set to True, returns a 4-tuple containing the key
    and the intermediate keys:

    ( signing_key, date_key, region_key, service_key )

    The intermediate keys can be used for testing against examples from
    Amazon.

    """
    init_key = ('AWS4' + secret_key).encode('utf-8')
    date_key = cls.sign_sha256(init_key, date)
    region_key = cls.sign_sha256(date_key, region)
    service_key = cls.sign_sha256(region_key, service)
    key = cls.sign_sha256(service_key, 'aws4_request')
    if intermediates:
        return (key, date_key, region_key, service_key)
    else:
        return key

def get_condition(list_of_dicts, condition):
    """input a list_of_dicts as found in policy['conditions'] and then iterate
    the dict keys in the list to get the condition."""
    for d in list_of_dicts:
        for k in d.keys():
            if condition in k:
                return d.get(k)

Next, I’m going to refer you to the server-side python example as shown at https://github.com/FineUploader/server-examples/blob/master/python/python3-flask-fine-uploader-s3/s3-sign-srv.py except, this example is for v2 signing, so we need to modify the sign_policy() and sign_headers() in that example to handle the AWS v4 signing requirements.

def sign_policy(policy):
    """ Sign (aws v4) and return the policy document for a simple upload.
    http://aws.amazon.com/articles/1434/#signyours3postform """
    policy_data = json.loads(policy)
    yyyymmdd = get_condition(policy_data['conditions'], 'x-amz-date')[:8]
    encoded_policy = base64.b64encode(
        bytearray(json.dumps(policy_data), 'utf-8'))
    signing_key = generate_key(AWS_CLIENT_SECRET_KEY, S3_REGION_NAME,
                                                's3', yyyymmdd)
    signature = sign_sha256(signing_key, encoded_policy.decode('utf-8'))

    return {
        'policy': encoded_policy.decode("utf-8"),
        'signature': binascii.hexlify(signature).decode("utf-8")
    }


def sign_headers(headers):
    """ Sign and return the headers for a chunked upload.
    The headers sent from fineupload have the majority of what is needed. But
    you must follow the process laid out in amazon docs to finish it.

    https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
    """
    # cut the first three lines from the headers from fine uploader and sign the remaining as hashed_canonical_request
    canonical_request = headers[70:]
    # split the headers so you can build the string_to_sign
    split_headers = headers.splitlines()
    # grab the date from the header
    yyyymmdd = split_headers[2][:8]
    # hash the canonical request and then return the base16 string
    hashed_canonical_request =  hash_sha256(canonical_request)
    # build the string_to_sign
    string_to_sign = split_headers[0] + 'n' + split_headers[1] + 'n' + split_headers[2] + 'n' + hashed_canonical_request
    # create the signing key
    signing_key = generate_key(AWS_CLIENT_SECRET_KEY, S3_REGION_NAME, 's3', yyyymmdd)
    # create the signature using the signing_key and string_to_sign
    signature = sign_sha256(signing_key, string_to_sign)

    return {
        'signature': binascii.hexlify(signature).decode("utf-8")
    }

If you’ve done it as laid out here and the stars are also aligned in your favor it should work. But if you still have errors, I found it immensely helpful to use the chrome devtools to review the xml data returned from amazon s3 regarding the 403 error. It will look like this (found on the NETWORK tab in a 403 error line which should be marked in red):

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>[your public key]</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256
20180725T101212Z
20180725/cn-north-1/s3/aws4_request
e7a7e92e17d8a3ac6228bb02139a499904db50a493ea6c336d847d4d94a5c320</StringToSign><SignatureProvided>2afc0bc1316732c9cd9bdc75c0aafdde70c3c96c0211991c610cf0c1bed33d71</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 31 38 30 37 32 35 54 31 30 31 32 31 32 5a 0a 32 30 31 38 30 37 32 35 2f 63 6e 2d 6e 6f 72 74 68 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 65 37 61 37 65 39 32 65 31 37 64 38 61 33 61 63 36 32 32 38 62 62 30 32 31 33 39 61 34 39 39 39 30 34 64 62 35 30 61 34 39 33 65 61 36 63 33 33 36 64 38 34 37 64 34 64 39 34 61 35 63 33 32 30</StringToSignBytes><CanonicalRequest>
POST
/39f0808e-3d0b-48d4-a3db-f171a3cb2943.mp4
uploads=
host:<your bucket>.s3.cn-north-1.amazonaws.com.cn
x-amz-acl:private
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20180725T101212Z
x-amz-meta-qqfilename:flux-field_pro_854x480p.mp4

host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-qqfilename
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</CanonicalRequest><CanonicalRequestBytes>50 4f 53 54 0a 2f 33 39 66 30 38 30 38 65 2d 33 64 30 62 2d 34 38 64 34 2d 61 33 64 62 2d 66 31 37 31 61 33 63 62 32 39 34 33 2e 6d 70 34 0a 75 70 6c 6f 61 64 73 3d 0a 68 6f 73 74 3a 61 6f 62 71 2e 73 33 2e 63 6e 2d 6e 6f 72 74 68 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 2e 63 6e 0a 78 2d 61 6d 7a 2d 61 63 6c 3a 70 72 69 76 61 74 65 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 31 38 30 37 32 35 54 31 30 31 32 31 32 5a 0a 78 2d 61 6d 7a 2d 6d 65 74 61 2d 71 71 66 69 6c 65 6e 61 6d 65 3a 66 6c 75 78 2d 66 69 65 6c 64 5f 70 72 6f 5f 38 35 34 78 34 38 30 70 2e 6d 70 34 0a 0a 68 6f 73 74 3b 78 2d 61 6d 7a 2d 61 63 6c 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 64 61 74 65 3b 78 2d 61 6d 7a 2d 6d 65 74 61 2d 71 71 66 69 6c 65 6e 61 6d 65 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35</CanonicalRequestBytes><RequestId>056AA9A01FBFF8FB</RequestId><HostId>RzQYHes10dAU3rrnhyDRRwN4NzuNxn3JrVcjlfK8NEqagFh0DZ0gkT56bMrYNwDTcU2iuZQohaY=</HostId></Error>

The canonical_request you created in the sign_headers() method must match what is shown in the s3 error <CanonicalRequest> part of the xml data. If it differs in any way then you will get an error. Similarly the string_to_sign in sign_headers() must also match the error response in the xml for <StringToSign>. Good luck.