Error while upgrading Airflow 1.11 to 1.15

Question:

Hi I’m planning to upgrade my Airflow version from 1.11 to 1.15 which is deployed in OpenShift. As there are very large numbers of DAG’s so I planned to upgrade in the bride release rather than going to Airflow 2.2

The error which I’m getting is most probably due to the fernet key:

ERROR: The `secret_key` setting under the webserver config has an insecure value - Airflow has 
failed safe and refuses to start. Please change this value to a new, per-environment,
 randomly generated string, for example using this command `openssl rand -hex 30`

Earlier I was using static Fernet Key and the YAML file is as follows:

apiVersion:v1
kind:Secret
metadata:
    name : airflow-secret
    namespace : CUSTOM_NAMESPACE
    labels: 
        app:airflow
type: Opaque
stringData:
   fernet-key: my_fernet_key




My Python Version : 3.8
My Airflow Webserver Config :

apiVersion: v1
kind: DeploymentConfig
metadata:
  name: airflow-webserver
  namespace: CUSTOM_NAMESPACE
  labels:
    app: airflow
spec:
  strategy: 
    type: Rolling
  trigger: 
    - type : ConfigChange
    - type : ImageChange
      ImageChangeParams: 
        automatic: true
        containerNames:
        - airflow-webserver
        from: 
          kind: ImageStreamTag
          namespace: CUSTOM_NAMESPACE
  replicas: 1
  revisionHistoryLimit : 10
  paused: false
  selector :
    app : airflow
    deploymentconfig : airflow-webserver
  template:
    metadata:
      labels:
        name: airflow-webserver
        app: airflow
        deploymentconfig : airflow-webserver
    spec: 
      volumes: 
      - name: airflow-dags
      persistentVolumeClaims:
        claimName: airflow-dags
    containers:
    - name:  airflow-webserver
      image:  airflow:latest
      resources:  
        limits:
          memory: 4Gi
      env:
        - name : FERNET_KEY
          valueFrom:
            secretKeyRef: 
              name: airflow-secrets
              key : fernet-key
        - name : SERVICE_ACCOUNT_NAME
          valueFrom:
            secretKeyRef:
              name: airflow-service-account
              key : service-account-name
      ports:
        - containerPort: 8080
          protocol: TCP

      volumeMounts:
        - name: airflow-dags
          mountPath: /opt/airflow/dags
        - name: airflow-logs
          mountPath: /opt/airflow/logs

My understanding is we need to somehow provide dynamic value in fernet key but for my case its static, Any Possible way to resolve the error.

Thank!

Asked By: peerpressure

||

Answers:

The main issue there was default value stored in airflow.cfg i.e.

secret_key = temporary_value

We can generate the secret_key by seeing the error message:

openssl rand -hex 30 

suppose the value is –> 94b9d6124ff2e9a5783d94dc7aa3641ebb8929bdbbf2f3989402f9e400ac

We need to put the value into the secret_key in airflow.cfg

secret_key = 94b9d6124ff2e9a5783d94dc7aa3641ebb8929bdbbf2f3989402f9e400ac
Answered By: peerpressure