airflow-taskflow

airflow.exceptions.AirflowException: 'branch_task_ids' must contain only valid task_ids Question: I have a dag which contains 1 custom task, 1 @task.branch task decorator and 1 taskgroup, inside the taskgroup I have multiple tasks that need to be triggered sequentially depending on the outcome of the @task.branch. PROCESS_BATCH_data_FILE = "batch_upload" SINGLE_data_FILE_FIRST_OPERATOR = "validate_data_schema_task" ENSURE_INTEGRITY_TASK = "provide_data_integrity_task" PROCESS_SINGLE_data_FILE = …

Restrict/Exclude specific logs from Airflow to Datadog Question: We are observing Airflow is sending large amount of logs to Datadog and we want to restrict/Reduce these logs by excluding logs from below tasks: pod_manager.py base.py base_aws.py logging_mixin.py Do we have any configuration settings where I can define this requirement? We have Airflow-2.0 running on Kubernetes. …