How should i store api call data for a webserver
Question:
I’m building a Flask web application. I use an API to retrieve data once per day. The data is returned as JSON. The JSON is stored in a file, and is sent to the client each time they access the site. The application uses an SQLite database to keep track of user information and user-associated data.
How should I go about storing the data received from the API? Would it be preferable to store it in the database since it is just updated once per day, or should I keep the response in a JSON format as I do currently?
Answers:
The real question you are asking is how should I store data. It doesn’t matter, nor does anybody care, where the data originates from: a message queue, an HTTP request, from the code itself. Here are the parameters for your problem:
- You have data.
- You want to know the best way to store the data.
- You are currently using a SQLite database for user information and associated data
- For this new data, you are currently storing it in a JSON file
Next you have to ask yourself the following questions:
- How much data are you storing?
- Is the data structured?
- Will I be querying the data by a specific field?
- Does the user need every part of the JSON file?
- How often will the data be requested?
I made a little flowchart to help you out:
I’m building a Flask web application. I use an API to retrieve data once per day. The data is returned as JSON. The JSON is stored in a file, and is sent to the client each time they access the site. The application uses an SQLite database to keep track of user information and user-associated data.
How should I go about storing the data received from the API? Would it be preferable to store it in the database since it is just updated once per day, or should I keep the response in a JSON format as I do currently?
The real question you are asking is how should I store data. It doesn’t matter, nor does anybody care, where the data originates from: a message queue, an HTTP request, from the code itself. Here are the parameters for your problem:
- You have data.
- You want to know the best way to store the data.
- You are currently using a SQLite database for user information and associated data
- For this new data, you are currently storing it in a JSON file
Next you have to ask yourself the following questions:
- How much data are you storing?
- Is the data structured?
- Will I be querying the data by a specific field?
- Does the user need every part of the JSON file?
- How often will the data be requested?
I made a little flowchart to help you out: