Python writing function output to a file
Question:
I have a function that simulates something and writes the resulting array to a text file.
For example,
def SimulationFunction(args):
#some simulation which results simOUT
f = open('results.txt','w')
f.write(str(simOUT) + 'n')
f.close()
I would like run the function on varying input arguments and save the outputs to the new lines of the same results.txt file.
For example,
for i in inputArgs:
SimulationFunction(i)
However, when I try to run this, the resulting text file only
contains the results from the last simulation run.
What am I missing here?
Any suggestions would be greatly appreciated!
Answers:
You must append to file not write to file.
Here is the fix:
f = open('results.txt','a')
'w' signifies that you are writing to the file (overwrites)
'a' signifies that you are appending to the file
If you want to learn more about different modes of opening file, visit:
http://www.tutorialspoint.com/python/python_files_io.htm
You must append to the file, rather than overwrite it.
According to Python Pocket Reference (Mark Lutz, O’rielly):
Mode is an optional string that specifies the mode in which the file is opened. It defaults to 'r' which means open for reading in text mode. Other common values are 'w' for writing (truncating the file if it already exists) and 'a' for appending.
Here are the modes for open():
(according to Tutorials Point)
r – Opens a file for reading only. The file pointer is placed at the beginning of the file. This is the default mode.
rb – Opens a file for reading only in binary format. The file pointer is placed at the beginning of the file. This is the default mode.
r+ – Opens a file for both reading and writing. The file pointer will be at the beginning of the file.
rb+ – Opens a file for both reading and writing in binary format. The file pointer will be at the beginning of the file.
w – Opens a file for writing only. Overwrites the file if the file exists. If the file does not exist, creates a new file for writing.
wb – Opens a file for writing only in binary format. Overwrites the file if the file exists. If the file does not exist, creates a new file for writing.
w+ – Opens a file for both writing and reading. Overwrites the existing file if the file exists. If the file does not exist, creates a new file for reading and writing.
wb+ – Opens a file for both writing and reading in binary format. Overwrites the existing file if the file exists. If the file does not exist, creates a new file for reading and writing.
a – Opens a file for appending. The file pointer is at the end of the file if the file exists. That is, the file is in the append mode. If the file does not exist, it creates a new file for writing.
ab – Opens a file for appending in binary format. The file pointer is at the end of the file if the file exists. That is, the file is in the append mode. If the file does not exist, it creates a new file for writing.
a+ – Opens a file for both appending and reading. The file pointer is at the end of the file if the file exists. The file opens in the append mode. If the file does not exist, it creates a new file for reading and writing.
ab+ – Opens a file for both appending and reading in binary format. The file pointer is at the end of the file if the file exists. The file opens in the append mode. If the file does not exist, it creates a new file for reading and writing.
Also, 'w' and 'a' will create a file if it does not exist, while 'r' will throw an exception of type IOError.
Using "a" (append) mode on your open() call as others suggest will solve your immediate problem, but then you have to worry about how to clear out the file for each run, or create it if it doesn’t exist, etc. You can do this by using "w" mode for the first function and "a" for subsequent ones, but this introduces a situation where they must be called in the right order. Which could come back to bite you in six months when you want them in a different order and have to chase down why only some of them are showing up in the file (because the one with “w” as the mode is now in the middle).
A better solution is to open the file outside the function and pass it in as an argument (perhaps using a with statement around the function calls so it gets closed automatically after). That not only avoids the need to worry about the mode in each of your functions, it avoids repeatedly opening and closing the file, which is an inefficiency (albeit a minor one on modern systems):
def SimulationFunction1(args, outfile):
# some simulation, which results in simOUT
outfile.write(str(simOUT) + 'n')
with open("my_output_file.txt", "w") as out:
SimulationFunction1(args, outfile=out)
SimulationFunction2(args, outfile=out)
SimulationFunction3(args, outfile=out)
Or better yet, just have the functions return the result and have the caller deal with writing them:
def SimulationFunction1(args):
# some simulation, which results in simOUT
return str(simOUT)
with open("my_output_file.txt", "w") as out:
out.write(SimulationFunction1(args) + `n`)
out.write(SimulationFunction2(args) + `n`)
out.write(SimulationFunction3(args) + `n`)
This latter approach illustrates separation of concerns. Each part of your code should be concerned with doing one thing: calculating a value, writing to a file, or whatever. Doing multiple things in a single “chunk” of code (function, class, whatever) makes it harder to understand, to maintain, and to reuse.
There’s a place where I could have (in fact, probably should have) separated concerns there, and didn’t… can you see it? 🙂
What I have found useful with my CSV files, that is, when I want to output a csv file directly from a function, I have done the following:
import pandas as pd #(allows me to use pd.to_csv)
#1. create a save path, location where I want to save my file
save_path = 'D:/results-output/fs.csv'
#2. create a small function that takes the csv data and path as arguments
def savefile(data_to_save, filepath):
return data_to_save.to_csv(filepath, index=False)
#3. call the function with the data and path arguments and the .csv file is outputted to the location in my save_path.
In this example, my data_to_save = patl
savefile(patl, save_path)
I have a function that simulates something and writes the resulting array to a text file.
For example,
def SimulationFunction(args):
#some simulation which results simOUT
f = open('results.txt','w')
f.write(str(simOUT) + 'n')
f.close()
I would like run the function on varying input arguments and save the outputs to the new lines of the same results.txt file.
For example,
for i in inputArgs:
SimulationFunction(i)
However, when I try to run this, the resulting text file only
contains the results from the last simulation run.
What am I missing here?
Any suggestions would be greatly appreciated!
You must append to file not write to file.
Here is the fix:
f = open('results.txt','a')
'w' signifies that you are writing to the file (overwrites)
'a' signifies that you are appending to the file
If you want to learn more about different modes of opening file, visit:
http://www.tutorialspoint.com/python/python_files_io.htm
You must append to the file, rather than overwrite it.
According to Python Pocket Reference (Mark Lutz, O’rielly):
Mode is an optional string that specifies the mode in which the file is opened. It defaults to
'r'which means open for reading in text mode. Other common values are'w'for writing (truncating the file if it already exists) and'a'for appending.
Here are the modes for open():
(according to Tutorials Point)
r– Opens a file for reading only. The file pointer is placed at the beginning of the file. This is the default mode.
rb– Opens a file for reading only in binary format. The file pointer is placed at the beginning of the file. This is the default mode.
r+– Opens a file for both reading and writing. The file pointer will be at the beginning of the file.
rb+– Opens a file for both reading and writing in binary format. The file pointer will be at the beginning of the file.
w– Opens a file for writing only. Overwrites the file if the file exists. If the file does not exist, creates a new file for writing.
wb– Opens a file for writing only in binary format. Overwrites the file if the file exists. If the file does not exist, creates a new file for writing.
w+– Opens a file for both writing and reading. Overwrites the existing file if the file exists. If the file does not exist, creates a new file for reading and writing.
wb+– Opens a file for both writing and reading in binary format. Overwrites the existing file if the file exists. If the file does not exist, creates a new file for reading and writing.
a– Opens a file for appending. The file pointer is at the end of the file if the file exists. That is, the file is in the append mode. If the file does not exist, it creates a new file for writing.
ab– Opens a file for appending in binary format. The file pointer is at the end of the file if the file exists. That is, the file is in the append mode. If the file does not exist, it creates a new file for writing.
a+– Opens a file for both appending and reading. The file pointer is at the end of the file if the file exists. The file opens in the append mode. If the file does not exist, it creates a new file for reading and writing.
ab+– Opens a file for both appending and reading in binary format. The file pointer is at the end of the file if the file exists. The file opens in the append mode. If the file does not exist, it creates a new file for reading and writing.
Also, 'w' and 'a' will create a file if it does not exist, while 'r' will throw an exception of type IOError.
Using "a" (append) mode on your open() call as others suggest will solve your immediate problem, but then you have to worry about how to clear out the file for each run, or create it if it doesn’t exist, etc. You can do this by using "w" mode for the first function and "a" for subsequent ones, but this introduces a situation where they must be called in the right order. Which could come back to bite you in six months when you want them in a different order and have to chase down why only some of them are showing up in the file (because the one with “w” as the mode is now in the middle).
A better solution is to open the file outside the function and pass it in as an argument (perhaps using a with statement around the function calls so it gets closed automatically after). That not only avoids the need to worry about the mode in each of your functions, it avoids repeatedly opening and closing the file, which is an inefficiency (albeit a minor one on modern systems):
def SimulationFunction1(args, outfile):
# some simulation, which results in simOUT
outfile.write(str(simOUT) + 'n')
with open("my_output_file.txt", "w") as out:
SimulationFunction1(args, outfile=out)
SimulationFunction2(args, outfile=out)
SimulationFunction3(args, outfile=out)
Or better yet, just have the functions return the result and have the caller deal with writing them:
def SimulationFunction1(args):
# some simulation, which results in simOUT
return str(simOUT)
with open("my_output_file.txt", "w") as out:
out.write(SimulationFunction1(args) + `n`)
out.write(SimulationFunction2(args) + `n`)
out.write(SimulationFunction3(args) + `n`)
This latter approach illustrates separation of concerns. Each part of your code should be concerned with doing one thing: calculating a value, writing to a file, or whatever. Doing multiple things in a single “chunk” of code (function, class, whatever) makes it harder to understand, to maintain, and to reuse.
There’s a place where I could have (in fact, probably should have) separated concerns there, and didn’t… can you see it? 🙂
What I have found useful with my CSV files, that is, when I want to output a csv file directly from a function, I have done the following:
import pandas as pd #(allows me to use pd.to_csv)
#1. create a save path, location where I want to save my file
save_path = 'D:/results-output/fs.csv'
#2. create a small function that takes the csv data and path as arguments
def savefile(data_to_save, filepath):
return data_to_save.to_csv(filepath, index=False)
#3. call the function with the data and path arguments and the .csv file is outputted to the location in my save_path.
In this example, my data_to_save = patl
savefile(patl, save_path)