Cloud Composer / Airflow start new task only when Cloud DataFusion task is really finished
Cloud Composer / Airflow start new task only when Cloud DataFusion task is really finished Question: I have the following task in Airflow (Cloud Composer) that triggers a Cloud DataFusion pipeline. The problem is: Airflow considers this task already a success when (within DataFusion) the DataProc cluster has been provisioned and the actual job has …
Total answers: 1