Google cloud composer get airflow webserver_id

Question

I have a GCP project my_project_id containing a composer instance my_project_id_cmpsr_id. In ordeer to get access Airflow rest API I need to retrieve the so called webserver_id. So the GCP airflow web server url is of the form {webserver-id}.appspot.com as specified here in the documentation

# This should be part of your webserver's URL:
# {tenant-project-id}.appspot.com
webserver_id = 'YOUR-TENANT-PROJECT'

Is it possible to retrieve webserver_id like fg8348538536e2df34-fd by project_id and composer_id?

Asked By: Some Name

||

Source

Answer 1

It is possible, you can go to your Airflow UI, than Admin -> Configuration, and search for base_url key, which is your webserver-id (without https:// and .appspot.com parts).

Another way to do so, is using the following command:

gcloud composer environments describe <ENVIRONMENT_NAME> --location <LOCATION>

And you will be able to see config: -> airflowUri variable.

I hope it helps.

Answered By: aga

Answer 2

You can get the values required with Python by running the code example on the Google docs and then modifiy it to return the webserver_id and client_id:

#!/usr/bin/env python
# coding: utf-8

import google.auth
import google.auth.transport.requests
import requests
import six.moves.urllib.parse

project_id = getenv('GCP_PROJECT_ID')
location = getenv('LOCATION')
composer_environment = getenv('COMPOSER_ID')


def get_airflow_details():
    # Authenticate with Google Cloud.
    # See: https://cloud.google.com/docs/authentication/getting-started
    credentials, _ = google.auth.default(
        scopes=['https://www.googleapis.com/auth/cloud-platform'])
    authed_session = google.auth.transport.requests.AuthorizedSession(
        credentials)

    environment_url = (
        'https://composer.googleapis.com/v1beta1/projects/{}/locations/{}'
        '/environments/{}').format(project_id, location, composer_environment)
    composer_response = authed_session.request('GET', environment_url)
    environment_data = composer_response.json()
    airflow_uri = environment_data['config']['airflowUri']
    # The Composer environment response does not include the IAP client ID.
    # Make a second, unauthenticated HTTP request to the web server to get the
    # redirect URI.
    redirect_response = requests.get(airflow_uri, allow_redirects=False)
    redirect_location = redirect_response.headers['location']

    # Extract the client_id query parameter from the redirect.
    parsed = six.moves.urllib.parse.urlparse(redirect_location)
    query_string = six.moves.urllib.parse.parse_qs(parsed.query)
    client_id = (query_string['client_id'][0])
    
    return client_id, airflow_uri



def main():

    get_airflow_details()
    
    client_id = (get_airflow_details()[0])
    airflow_uri = (get_airflow_details()[1])
    print(client_id)
    print(airflow_uri)

    return 'Script has run without errors !!'

if (__name__ == "__main__"):
    main()

Answered By: Hutch

Answer 3

gcloud composer environments describe "$COMPOSER_ENV" 
    --location "$LOCATION" 
    --project "$PROJECT" 
    --format "value(config.airflowUri)"

Answered By: p13rr0m

Google cloud composer get airflow webserver_id

Question:

Answers: