What should I do when Airflow task gets queued but never start running?
Question:
This is a simple question.
My Airflow sometimes gets queued but never starts running.
Does it mean I have an error in my code?
What should I do?
My environment:
- Python 3.7
- apache-airflow==2.2.2
- aws-mwaa-local-runner
Answers:
According to the discussion, we should at least do the following.
- check that the pool assigned to the DAG exists and it has empty slots
- raise an issue in the Airflow repo with more details about your setup & DAG
This is a simple question.
My Airflow sometimes gets queued but never starts running.
Does it mean I have an error in my code?
What should I do?
My environment:
- Python 3.7
- apache-airflow==2.2.2
- aws-mwaa-local-runner
According to the discussion, we should at least do the following.
- check that the pool assigned to the DAG exists and it has empty slots
- raise an issue in the Airflow repo with more details about your setup & DAG