Airflow: unable to configure handler "processor"

Question

I’m very very new to Apache-airflow, just started a course in udemy (this course) .

We have recieved YAML file and were asked to follow instructions in order to install airflow.
I have been able to install airflow, I beleive, because I have gotten any error while doing the next steps:

create a new file new_file.env (Visual Studio)
add these lines inside new file and save it :

AIRFLOW_IMAGE_NAME=apache/airflow:2.3.0
AIRFLOW_UID=50000

open the terminal and run

docker-compose up -d

Then when I tried to open localhost:8080 nothing was opened.

When I checked the containers, using

docker-compose ps

I have seen that some of the containers are not healthy.

docker logs materials_name_of_the_container #here I inserted every tome different container name

The most common error was "unable to configure handler "processor"

I have seen here post with similar problem that recommened to use pip to install Azure ,and I have tried to print this in the terminal:

pip install 'apache-airflow[azure_blob_storage,azure_data_lake,azure_cosmos,azure_container_instances]

#also tried with one more '

pip install 'apache-airflow[azure_blob_storage,azure_data_lake,azure_cosmos,azure_container_instances]'

I got an error that pip is not recognized:

My goal is to complete the installation , and "on -the-way" to understand why these errors happened. I beleive the the YAML file is well (because it’s an organized course), but no idea where or what happenned inside the contaienrs, and where these containers are located, so any "dummy" explaination for beginner is welcome.

Asked By: Reut

||

Source

Answer 1

There are two reasons why this shouldn’t work:

PIP Installation path is not added to the system variables: If you have installed python through prompt then you need to configure PIP path manually.
Wrong system path added: If you have installed multiple versions of python, which can affect the installation path, you need to fix the files manually.

More information here

Maybe you can try :

python -m pip install [name]

instead of pip alone.

OR you can add pip to a path:

import os
import sys
pipPath = f'{os.path.dirname(sys.executable)}\Scripts'
os.system(f'setx PATH "%PATH%;{pipPath}"')

Source

EDIT: After reading the OP’s answer, I think this is the reason why administrator is required:

Area	Task	For more information
Installation	Install or modify Visual Studio	Install Visual Studio, Modify Visual Studio
	Install, update, or remove local Help content	Install and manage local Help content

This may affect the usage of pip and it’s installation!

Microsoft docs – User permissions and visual studio

Answered By: DialFrost

Answer 2

In the end I could solve this issue , after reading this troubleshooting page. I was trying to configure the users&groups, then I have realized that for some reason, my user is not the administrator. I have changed it, uninstall everything (docker+visual studio) and then installed it again , and- yayyy! it worked and this error didn’t appear.

so the steps I did:
~the foirst part of changing to administrator- I have done yesterday and I don’t remember 100% how, I think it was this, if it’s wrong please let me know…

Enter the users&groups – that was not available in my windows 10, if I remember correct, I have pressed "windows" +"R" on the keyboard, and then I typed "netplwiz" and pressed enter.

2.double click on the username, and this tab was opened:

select administraor (the screenshow is after the fix so maybe it doesn’t have the docker options)

4. restart the computer
5. uninstalled docker and visual studio
6. installed docker and visual studio again
7. docker-compose up -d ….
then after few minutes it worked.

Thanks for everyone who tried to help me 🙂

let me know if there is somethin g to improve in this answer.

Answered By: Reut

Airflow: unable to configure handler "processor"

Question:

Answers: