How to use two APIs to get the response of an endpoint once it processed?
Question:
I have two APIs: triggerAPI and triggerAPIResult. When I hit the first one, it would trigger a process which could take a few minutes to return the response. The second API is used to check if the process is successfully finished or not.
Therefore, when the second API returns true, that means now the response from the first API is the desired output. The second API response is very important since when the first API is still processing, it would return meaningless data until actually finished. Another thing is that the triggerAPIResult API should get triggered every minute for 10 minutes to constantly check the result. How could you I implement this in Python?
Answers:
You need to implement long running operations. This is an implementation strategy used by fe. Google (in their GCP APIs), IBM, and other big companies.
The principle is quite simple.
- Do a request to the
triggerAPI and immediately return a unique operation ID.
- Store this ID somewhere and have an
is_done value tied to it which is set to False.
- Have whatever logic was triggered do its thing. Once it’s done, update the operation and set the
is_done value to True. Store the result of the operation somewhere.
- Call the
triggerAPIResult and have logic to check the state of the operation. Return the actual processed data when done, otherwise return something like still working.
Note that the actual implementation can be a bit complicated depending on the tech used. I would go for microservices in this scenario to avoid having to use threads or async.
- External API with the two endpoints you mentioned.
- Internal API that does the actual data processing.
- NoSQL backend that stores the operation status’ and the result of the data processing.
I have two APIs: triggerAPI and triggerAPIResult. When I hit the first one, it would trigger a process which could take a few minutes to return the response. The second API is used to check if the process is successfully finished or not.
Therefore, when the second API returns true, that means now the response from the first API is the desired output. The second API response is very important since when the first API is still processing, it would return meaningless data until actually finished. Another thing is that the triggerAPIResult API should get triggered every minute for 10 minutes to constantly check the result. How could you I implement this in Python?
You need to implement long running operations. This is an implementation strategy used by fe. Google (in their GCP APIs), IBM, and other big companies.
The principle is quite simple.
- Do a request to the
triggerAPIand immediately return a unique operation ID. - Store this ID somewhere and have an
is_donevalue tied to it which is set toFalse. - Have whatever logic was triggered do its thing. Once it’s done, update the operation and set the
is_donevalue toTrue. Store the result of the operation somewhere. - Call the
triggerAPIResultand have logic to check the state of the operation. Return the actual processed data when done, otherwise return something likestill working.
Note that the actual implementation can be a bit complicated depending on the tech used. I would go for microservices in this scenario to avoid having to use threads or async.
- External API with the two endpoints you mentioned.
- Internal API that does the actual data processing.
- NoSQL backend that stores the operation status’ and the result of the data processing.