Downloading a file from S3 to local machine using boto3
Question:
I’m trying to use a Python script to automate downloading a file from AWS S3 to my local machine. The Python script itself is hosted on Ubuntu (AWS EC2 instance), so it’s not recognizing a directory on my local machine.
Here’s my code:
import os
import boto3
from boto3.session import Session
print("this script downloads the file from s3 to local machine")
s3 = boto3.resource('s3')
BUCKET_NAME = 'sfbucket.myBucket'
KEY = 'sf_events.json'
s3.Bucket(BUCKET_NAME).download_file(KEY, '/Users/Documents/my_file.json')
print('end')
However, this gives me the following error:
FileNotFoundError: [Errno 2] No such file or directory: ‘/Users/Documents/my_file.json.CDC5FEf4’
What am I doing wrong? If I replace the output directory to /home/ubuntu/ it works fine, but I want the file on my local machine.
Answers:
The script has to run on your local Windows machine not on EC2 instance.
Alternatively, you can simply use aws cli
aws s3 cp s3://sfbucket.myBucket/sf_events.json /Users/Documents/my_file.json
As you are trying to download the file using the EC2 instance, this machine doesn’t know your local path /Users/Documents/my_file.json
You have two options:
-
Run this script directly in your local machine.
In this case, you have to run this script in your local machine and make sure you have access to this bucket.
-
Run this script in EC2 Instance and copy.
In this case, you have to download the file to somewhere /home/ubuntu/ and then after in your local machine make a copy from EC2.
You can use [SCP][1]
You can install the SCP server in your local machine but the problem is that your local machine probably doesn’t have a private IP, so every time that your local IP change you have to update the script. Maybe you should think about to do whatever you need to do with this file directly on EC2 Instance or if it’s a real manual thing, sending via email, maybe?
I’m trying to use a Python script to automate downloading a file from AWS S3 to my local machine. The Python script itself is hosted on Ubuntu (AWS EC2 instance), so it’s not recognizing a directory on my local machine.
Here’s my code:
import os
import boto3
from boto3.session import Session
print("this script downloads the file from s3 to local machine")
s3 = boto3.resource('s3')
BUCKET_NAME = 'sfbucket.myBucket'
KEY = 'sf_events.json'
s3.Bucket(BUCKET_NAME).download_file(KEY, '/Users/Documents/my_file.json')
print('end')
However, this gives me the following error:
FileNotFoundError: [Errno 2] No such file or directory: ‘/Users/Documents/my_file.json.CDC5FEf4’
What am I doing wrong? If I replace the output directory to /home/ubuntu/ it works fine, but I want the file on my local machine.
The script has to run on your local Windows machine not on EC2 instance.
Alternatively, you can simply use aws cli
aws s3 cp s3://sfbucket.myBucket/sf_events.json /Users/Documents/my_file.json
As you are trying to download the file using the EC2 instance, this machine doesn’t know your local path /Users/Documents/my_file.json
You have two options:
-
Run this script directly in your local machine.
In this case, you have to run this script in your local machine and make sure you have access to this bucket. -
Run this script in EC2 Instance and copy.
In this case, you have to download the file to somewhere/home/ubuntu/and then after in your local machine make a copy from EC2.
You can use [SCP][1]
You can install the SCP server in your local machine but the problem is that your local machine probably doesn’t have a private IP, so every time that your local IP change you have to update the script. Maybe you should think about to do whatever you need to do with this file directly on EC2 Instance or if it’s a real manual thing, sending via email, maybe?