How to store randomly generated lists in python
Question:
I am a mathematician with limited programming experience, but am building a neural network to generate curves with certain properties. I’ve built a training set which looks like a list of 1000 randomly generated numbers between -1 and 1 as follows:
import random
N = 1000 # Number of points in training set
Training_set = []
for i in range(N):
Training_set.append(random.uniform(-1, 1))
This works just fine, but I’d like to save the list Training_set so that I can modify the script and see how the resulting cost function changes with respect to the same training set. Currently, a new training set is generated every time I run the script. I tried writing the above code in a separate python file and importing it to the file where I’m running analysis on the neural network, but to my dismay, a new training set is generated every time even with imports. Copy-pasting is the simplest option in terms of saving randomly generated data, but with such a large list, that seems a bit ridiculous to me (especially if I want to keep scaling up N).
How can I save the data generated in one file somewhere else in a way that it can be called upon again later? This seems like a fairly basic and fundamental thing, I’m sure there’s a lot of references out there that address this, but I didn’t even know how to search for it. So if someone can point me in the right direction (either explicitly or with a reference), it would be greatly appreciated.
Answers:
If you want to have the same values everytime you run the program, you can use random.seed.
import random
random.seed(1) # can be any positive int value
N = 1000 # Number of points in training set
Training_set = []
for i in range(N):
Training_set.append(random.uniform(-1, 1))
Another option is to export your data to a .csv or .txt, for example, and then read from the file.
As a mathematician I suggest you can familiar with numpy which will help you overcome many of python’s in-built functions limitations. Using numpy you can try:
import numpy as np
np.savetxt('test.out', np.random.uniform(-1,1,1000), delimiter=',')
np.savetxt allows to save the output of 1000 random np.random.uniform() samples between -1 and 1
Edit: After re-reading your question, indeed it is more important to set a seed rather than freezing your data.
An easy solution is to write to a text file so you can compare view and compare the values, and you can also read it back into the code.
import random
N = 1000 # Number of points in training set
Training_set = []
for i in range(N):
Training_set.append(random.uniform(-1, 1))
# Write new values to a text file
with open('Training_set.txt', 'a') as f:
f.writelines(f"{Training_set}n")
# read from a text file
with open('Training_set.txt', 'r') as f:
training_set_output = f.readlines()
print(training_set_output)
I am a mathematician with limited programming experience, but am building a neural network to generate curves with certain properties. I’ve built a training set which looks like a list of 1000 randomly generated numbers between -1 and 1 as follows:
import random
N = 1000 # Number of points in training set
Training_set = []
for i in range(N):
Training_set.append(random.uniform(-1, 1))
This works just fine, but I’d like to save the list Training_set so that I can modify the script and see how the resulting cost function changes with respect to the same training set. Currently, a new training set is generated every time I run the script. I tried writing the above code in a separate python file and importing it to the file where I’m running analysis on the neural network, but to my dismay, a new training set is generated every time even with imports. Copy-pasting is the simplest option in terms of saving randomly generated data, but with such a large list, that seems a bit ridiculous to me (especially if I want to keep scaling up N).
How can I save the data generated in one file somewhere else in a way that it can be called upon again later? This seems like a fairly basic and fundamental thing, I’m sure there’s a lot of references out there that address this, but I didn’t even know how to search for it. So if someone can point me in the right direction (either explicitly or with a reference), it would be greatly appreciated.
If you want to have the same values everytime you run the program, you can use random.seed.
import random
random.seed(1) # can be any positive int value
N = 1000 # Number of points in training set
Training_set = []
for i in range(N):
Training_set.append(random.uniform(-1, 1))
Another option is to export your data to a .csv or .txt, for example, and then read from the file.
As a mathematician I suggest you can familiar with numpy which will help you overcome many of python’s in-built functions limitations. Using numpy you can try:
import numpy as np
np.savetxt('test.out', np.random.uniform(-1,1,1000), delimiter=',')
np.savetxt allows to save the output of 1000 random np.random.uniform() samples between -1 and 1
Edit: After re-reading your question, indeed it is more important to set a seed rather than freezing your data.
An easy solution is to write to a text file so you can compare view and compare the values, and you can also read it back into the code.
import random
N = 1000 # Number of points in training set
Training_set = []
for i in range(N):
Training_set.append(random.uniform(-1, 1))
# Write new values to a text file
with open('Training_set.txt', 'a') as f:
f.writelines(f"{Training_set}n")
# read from a text file
with open('Training_set.txt', 'r') as f:
training_set_output = f.readlines()
print(training_set_output)