How can one use HashiCorp Vault in Airflow?


I am starting to use Apache Airflow and I am wondering how to effectively make it use secrets and passwords stored in Vault. Unfortunately, search does not return meaningful answers beyond a yet-to-be-implemented hook in Airflow project itself.

I can always use Python’s hvac module to generically access Vault from PythonOperator but I was wondering if there is any better way or a good practice (e.g. maybe an Airflow plugin I missed).

Asked By: sophros



Airflow >=1.10.10 supports Secrets Backends and supports getting Airflow Variables and Connections from Hashicorp Vault.

More Details in Airflow Docs:

If you want to test it locally check the tutorial at

Set the following config in airflow.cfg, update based on your environment:

backend = airflow.contrib.secrets.hashicorp_vault.VaultBackend
backend_kwargs = {"connections_path": "connections", "variables_path": "variables", "mount_point": "airflow", "url": ""}

Example DAG to test the integration:

from airflow import DAG
from airflow.operators.python_operator import PythonOperator
from datetime import datetime
from airflow.hooks.base_hook import BaseHook

def get_secrets(**kwargs):
    conn = BaseHook.get_connection(kwargs['my_conn_id'])
    print(f"Password: {conn.password}, Login: {conn.login}, URI: {conn.get_uri()}, Host: {}")

with DAG('example_secrets_dags', start_date=datetime(2020, 1, 1), schedule_interval=None) as dag:

    test_task = PythonOperator(
        op_kwargs={'my_conn_id': 'smtp_default'},
Answered By: kaxil

Additional things to keep eye on If you are new to Vault:

Validate that your airflow.cfg backend_kwargs parameter mount_point really exists in vault:
vault secrets list

Try your DAG code above with command line first before trying it in web interface
airflow test example_secrets_dags test-task 2020-06-04

Answered By: Erkko

How to config with SSL vault
backend = airflow.contrib.secrets.hashicorp_vault.VaultBackend
backend_kwargs = {"connections_path": "connections", "variables_path": "variables", "mount_point": "airflow", "url": ""}

Answered By: Minh Huy