What is the best way to store data and when to use databases?
Question:
As a newbie looking to create my first program soon, I don’t have much experience at all when it comes to saving data and accessing that data.
My question is, what kind of data do you store locally? What kind of data would you store on a database? Why would I choose to use a database over locally stored files? (I understand this last question sounds silly, I’m just trying to get a deeper understanding).
If I write a program that requires the user to input a file path directory to store data locally, would I store the file path directory the user inputted in a regular text file to read later? Or are there specific file types to store data like that in? I’ve read about pickling data, is this a reliable way to store data?
(This question may not belong on here, I apologize if this is the case, I’m not sure where else to ask this question)
Answers:
The answer will very much depend on your intended use of the data.
The intended use of the data, the frequency of read/write/delete access all change the answer as well, so you really do need to elaborate on what your intents are.
Consider the following:
- In RAM storage in Python with lists / dictionaries etc. is useful for small amounts of temporary working data, but not suitable for long term storage.
- SQLITE is an embedded database which is suitable for non-parallel usage if you need to store relational data. It is saved to a local file.
- A database server is useful if you need multiple processes accessing the data simultaneously.
You will likely want to use different ways of storage depending on whether you are storing configuration data, where you might prefer local storage with text, csv, json, ini or toml files.
In cases where you need a database server, you will likely know you need it by then, so without any other details, I would not offer more possibilities.
Using database optional if you are new to programming.
You should stick to the basics and learn to be comfortable with coding itself; solving problems and all. But, once you are actually building programs that can be used by multiple users, then it’s a good idea to start implementing databases into your application.
As a newbie looking to create my first program soon, I don’t have much experience at all when it comes to saving data and accessing that data.
My question is, what kind of data do you store locally? What kind of data would you store on a database? Why would I choose to use a database over locally stored files? (I understand this last question sounds silly, I’m just trying to get a deeper understanding).
If I write a program that requires the user to input a file path directory to store data locally, would I store the file path directory the user inputted in a regular text file to read later? Or are there specific file types to store data like that in? I’ve read about pickling data, is this a reliable way to store data?
(This question may not belong on here, I apologize if this is the case, I’m not sure where else to ask this question)
The answer will very much depend on your intended use of the data.
The intended use of the data, the frequency of read/write/delete access all change the answer as well, so you really do need to elaborate on what your intents are.
Consider the following:
- In RAM storage in Python with lists / dictionaries etc. is useful for small amounts of temporary working data, but not suitable for long term storage.
- SQLITE is an embedded database which is suitable for non-parallel usage if you need to store relational data. It is saved to a local file.
- A database server is useful if you need multiple processes accessing the data simultaneously.
You will likely want to use different ways of storage depending on whether you are storing configuration data, where you might prefer local storage with text, csv, json, ini or toml files.
In cases where you need a database server, you will likely know you need it by then, so without any other details, I would not offer more possibilities.
Using database optional if you are new to programming.
You should stick to the basics and learn to be comfortable with coding itself; solving problems and all. But, once you are actually building programs that can be used by multiple users, then it’s a good idea to start implementing databases into your application.