Copy file from ephemeral docker container to host
Question:
I want to copy a file generated by a docker container, and store inside it to my local host. What is a good way of doing that? The docker container is ephemeral (i.e. it runs for a very short time and then stops.)
I am working with the below mentioned scripts:
Python (script.py) which generates and saves a file titled read.txt.
with open('read.txt', 'w') as file:
lst = ['Somen',
'randomn',
'sentencen']
file.writelines("% sn" % words for words in lst)
I use the below Dockerfile:
FROM python:3.9
WORKDIR /app
RUN pip install --upgrade pip
COPY . /app/
RUN pip install --requirement /app/requirements.txt
CMD ["python", "/app/script.py.py"]
Below is my folder structure:
- local
- folder1
- script.py
- requirements.txt
- Dockerfile
- folder2
Till now, I have managed to successfully build a docker container using:
docker build --no-cache -t test:v1 .
When I run this docker container inside /local/folder1/ using the below command, I get the desired file, i.e. read.txt inside /local/folder1/
docker run -v /local/folder1/:/app/ test:v1
But, when I run docker run -v /local/folder2/:/app/ test:v1 inside /local/folder2/, I do not see read.txt inside /local/folder2/ and I get the below message.
python: can't open file '/app/script.py': [Errno 2] No such file or directory
I want to be able to get read.txt inside /local/folder2/ when I run the docker container test:v1 inside /local/folder2/. How can I do it? I want to be able to do it without copying the contents of /local/folder1/ inside /local/folder2/.
The docker container is ephemeral (i.e. it runs for a very short time and then stops.) Hence answers given in this and this Stackoverflow posts, which focus on docker cp have not worked for me.
Time of running of the abovementioned container is not very essential. Even if a workable
solution increases the time of running of a container, that is okay.
Answers:
The problem is: there’s overlap between a mounted path and image’s already existing filesystem, docker will choose to not override your host’s (laptop etc) filesystem. So in folder1 you were actually running the python code from your host’s filesystem. To verify this you can change the script.py (add a new output line for file etc) and without building a new image, run the old one. You’ll have different output.
How to fix:
Simplest way
Just create another directory for output:
FROM python:3.9
WORKDIR /app
RUN mkdir /output
RUN pip install --upgrade pip
COPY . /app/
RUN pip install --requirement /app/requirements.txt
CMD ["python", "/app/script.py"]
Change python script to write to /output/read.txt
with open('/output/read.txt', 'w') as file:
lst = ['Someen',
'randomn',
'sentencen']
file.writelines("% sn" % words for words in lst)
More generic ways
You can choose output directory from environment variable and set via ENV in Dockerfile. Maybe use of ARG can be good use case for you.
Disclaimer for the obvious simplest way
Usage of /tmp may cause problem that you never thought, some programs use sentinel files instead of IPC because you /tmp is mounted to host it’ll override initial state of /tmp (I’m not sure it’ll be ever problem but it may…)
Since the container is ephemeral, you may want to run the container and keep it running. And then copy the file.
Try the below steps:
- Keep the container running.
docker run -it --name temp test:v1 sh
Here temp is a name of a temporary container.
- Exit the container.
You would be inside an interactive shell. Exit using exit command.
- Copy file from temp container to local host.
docker cp temp:/app/read.txt /local/folder2/.
- Remove the temporary container.
docker rm temp
I want to copy a file generated by a docker container, and store inside it to my local host. What is a good way of doing that? The docker container is ephemeral (i.e. it runs for a very short time and then stops.)
I am working with the below mentioned scripts:
Python (script.py) which generates and saves a file titled read.txt.
with open('read.txt', 'w') as file:
lst = ['Somen',
'randomn',
'sentencen']
file.writelines("% sn" % words for words in lst)
I use the below Dockerfile:
FROM python:3.9
WORKDIR /app
RUN pip install --upgrade pip
COPY . /app/
RUN pip install --requirement /app/requirements.txt
CMD ["python", "/app/script.py.py"]
Below is my folder structure:
- local
- folder1
- script.py
- requirements.txt
- Dockerfile
- folder2
Till now, I have managed to successfully build a docker container using:
docker build --no-cache -t test:v1 .
When I run this docker container inside /local/folder1/ using the below command, I get the desired file, i.e. read.txt inside /local/folder1/
docker run -v /local/folder1/:/app/ test:v1
But, when I run docker run -v /local/folder2/:/app/ test:v1 inside /local/folder2/, I do not see read.txt inside /local/folder2/ and I get the below message.
python: can't open file '/app/script.py': [Errno 2] No such file or directory
I want to be able to get read.txt inside /local/folder2/ when I run the docker container test:v1 inside /local/folder2/. How can I do it? I want to be able to do it without copying the contents of /local/folder1/ inside /local/folder2/.
The docker container is ephemeral (i.e. it runs for a very short time and then stops.) Hence answers given in this and this Stackoverflow posts, which focus on docker cp have not worked for me.
Time of running of the abovementioned container is not very essential. Even if a workable
solution increases the time of running of a container, that is okay.
The problem is: there’s overlap between a mounted path and image’s already existing filesystem, docker will choose to not override your host’s (laptop etc) filesystem. So in folder1 you were actually running the python code from your host’s filesystem. To verify this you can change the script.py (add a new output line for file etc) and without building a new image, run the old one. You’ll have different output.
How to fix:
Simplest way
Just create another directory for output:
FROM python:3.9
WORKDIR /app
RUN mkdir /output
RUN pip install --upgrade pip
COPY . /app/
RUN pip install --requirement /app/requirements.txt
CMD ["python", "/app/script.py"]
Change python script to write to /output/read.txt
with open('/output/read.txt', 'w') as file:
lst = ['Someen',
'randomn',
'sentencen']
file.writelines("% sn" % words for words in lst)
More generic ways
You can choose output directory from environment variable and set via ENV in Dockerfile. Maybe use of ARG can be good use case for you.
Disclaimer for the obvious simplest way
Usage of /tmp may cause problem that you never thought, some programs use sentinel files instead of IPC because you /tmp is mounted to host it’ll override initial state of /tmp (I’m not sure it’ll be ever problem but it may…)
Since the container is ephemeral, you may want to run the container and keep it running. And then copy the file.
Try the below steps:
- Keep the container running.
docker run -it --name temp test:v1 sh
Here temp is a name of a temporary container.
- Exit the container.
You would be inside an interactive shell. Exit using exit command.
- Copy file from temp container to local host.
docker cp temp:/app/read.txt /local/folder2/.
- Remove the temporary container.
docker rm temp