Python – writing and reading from a temporary file

Question

I am trying to create a temporary file that I write in some lines from another file and then make some objects from the data. I am not sure how to find and open the temp file so I can read it. My code:

with tempfile.TemporaryFile() as tmp:
    lines = open(file1).readlines()
    tmp.writelines(lines[2:-1])

dependencyList = []

for line in tmp:
    groupId = textwrap.dedent(line.split(':')[0])
    artifactId = line.split(':')[1]
    version = line.split(':')[3]
    scope = str.strip(line.split(':')[4])
    dependencyObject = depenObj(groupId, artifactId, version, scope)
    dependencyList.append(dependencyObject)
tmp.close()

Essentially I just want to make a middleman temporary document to protect against accidentally overwriting a file.

Asked By: fractalflame

||

Source

Answer 1

As per the docs, the file is deleted when the TemporaryFile is closed and that happens when you exit the with clause. So… don’t exit the with clause. Rewind the file and do your work in the with.

with tempfile.TemporaryFile() as tmp:
    lines = open(file1).readlines()
    tmp.writelines(lines[2:-1])
    tmp.seek(0)

    for line in tmp:
        groupId = textwrap.dedent(line.split(':')[0])
        artifactId = line.split(':')[1]
        version = line.split(':')[3]
        scope = str.strip(line.split(':')[4])
        dependencyObject = depenObj(groupId, artifactId, version, scope)
        dependencyList.append(dependencyObject)

Answered By: tdelaney

Answer 2

You’ve got a scope problem; the file tmp only exists within the scope of the with statement which creates it. Additionally, you’ll need to use a NamedTemporaryFile if you want to access the file later outside of the initial with (this gives the OS the ability to access the file). Also, I’m not sure why you’re trying to append to a temporary file… since it won’t have existed before you instantiate it.

Try this:

import tempfile

tmp = tempfile.NamedTemporaryFile()

# Open the file for writing.
with open(tmp.name, 'w') as f:
    f.write(stuff) # where `stuff` is, y'know... stuff to write (a string)

...

# Open the file for reading.
with open(tmp.name) as f:
    for line in f:
        ... # more things here

Answered By: Pierce Darragh

Answer 3

In case the file needs to be opened a second time, e.g. read by a different process this might cause trouble on Windows OS:

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).

Hence a safe solution is to create a temporary directory instead and then manually create a file therein:

import os.path
import tempfile

with tempfile.TemporaryDirectory() as td:
    f_name = os.path.join(td, 'test')
    with open(f_name, 'w') as fh:
        fh.write('<content>')
    # Now the file is written and closed and can be used for reading.

Answered By: a_guest

Answer 4

I just want to make a middleman temporary document to protect against accidentally overwriting a file.

Hopefully you’re opening the file in read mode and avoiding arbitrary os.remove() calls, so in the common case there shouldn’t be much chance of an accident.

That said, if the normal protections are off for a good reason, you can back up the file by creating a temp file and copying the original file’s contents to it with shutil:

import shutil
import tempfile


orig_path = "hello.txt"

# make a sample file
with open(orig_path, "w") as f:
    f.write("hello world")

with tempfile.NamedTemporaryFile() as temp:
    shutil.copy2(orig_path, temp.name)

    # or optionally write to the file:
    #temp.write("hello world".encode("utf-8"))
    #temp.flush()
    #temp.seek(0)

    # temp will be available in this block
    print(temp.read().decode("utf-8")) # => hello world

# temp will be deleted after the block

Answered By: ggorlen

Python – writing and reading from a temporary file

Question:

Answers: