How to get debug logs from boto3 in a local script?

Question

I have a local script that lists buckets:

import boto3
s3_client = boto3.client('s3')

for bucket in s3_client.list_buckets()["Buckets"]:
    print(bucket['Name'])

when I execute it locally, it does just that. Now if I execute this code as a lambda on AWS and set the log level to DEBUG, like so:

import boto3
import logging

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

s3_client = boto3.client('s3', endpoint_url="http://localhost:4566")

def lambda_handler(event, context):
    s3_client.list_buckets()
    return { "statusCode": 200 }

I get detailed logs such as the headers of the HTTP request that is sent to S3.

But If I add these lies to my local script nothing changes. I’ve tried:

import boto3
import logging

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

s3_client = boto3.client('s3')

for bucket in s3_client.list_buckets()["Buckets"]:
    print(bucket['Name'])

and

import boto3
import logging

logging.getLogger('boto3').setLevel(logging.DEBUG)
logging.getLogger('botocore').setLevel(logging.DEBUG)
logging.getLogger('s3transfer').setLevel(logging.DEBUG)

s3_client = boto3.client('s3')

for bucket in s3_client.list_buckets()["Buckets"]:
    print(bucket['Name'])

as suggested here and in both cases I get no logs.
How can I get my local script the show the logs for what boto3 does under the hood?

Asked By: peer

||

Source

Answer 1

Adding one log line of my own seems to trigger the botocore logs being printed as well. I have no idea why though.

import boto3
import logging

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

logging.debug("hello?")

s3_client = boto3.client('s3')

for bucket in s3_client.list_buckets()["Buckets"]:
    print(bucket['Name'])

Answered By: peer

Answer 2

This seems to work for me:

import boto3
import logging

boto3.set_stream_logger('', logging.DEBUG)

s3_client = boto3.client('s3')

for bucket in s3_client.list_buckets()["Buckets"]:
    print(bucket['Name'])

Spews out a whole bunch of stuff. From this doc

Answered By: Daniel Farrell

How to get debug logs from boto3 in a local script?

Question:

Answers: