Firebase credentials as Python environment variables: Could not deserialize key data
Question:
I’m developing a Python web app with a Firestore realtime database using the firebase_admin library. The Firestore key comes in form of a .json file containing 10 variables. However, I want to store some of these variables as environment variables so they are not visible publicly. So, I don’t use a Firebase SDK .json file, but I create my own dictionary with the elements of this file. The dictionary looks like this and everything has been directly copied from the .json file:
my_credentials = {
"type": "service_account",
"project_id": "bookclub-b2db5",
"private_key_id": os.environ.get("PRIVATE_KEY_ID"),
"private_key": os.environ.get("PRIVATE_KEY"),
"client_email": os.environ.get("CLIENT_EMAIL"),
"client_id": os.environ.get("CLIENT_ID"),
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": os.environ.get("AUTH_PROVIDER_X509_CERT_URL"),
"client_x509_cert_url": os.environ.get("AUTH_PROVIDER_X509_CERT_URL")
}
I have set the private key as a PRIVATE_KEY environment variable.
The private key looks roughly like this (the characters here are made up):
"-----BEGIN PRIVATE KEY-----nEW8nYP9T840Sb8tQMinhZ(...MORE CHARACTERS HERE...)EW8nYP9T840Sb8tQMi/EW8nYP9T840Sb8tQMiEW8nYP9T840Sb8tQMi/EW8nYP9T840Sb8tQMi=n-----END PRIVATE KEY-----n"
Then, I try to create a Firebase Certificate from these credentials and initialize the application
cred = firebase_admin.credentials.Certificate(my_credentials)
firebase_admin.initialize_app(cred, {'databaseURL': 'https://myapp.firebasedatabase.app/'})
However, it turns out that the PRIVATE_KEY environment variable is not readable (cannot be deserialized), and it throws an error:
Traceback (most recent call last):
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagesfirebase_admincredentials.py", line 96, in __init__
self._g_credential = service_account.Credentials.from_service_account_info(
File "C:UsersjaremAppDataLocalProgramsPythonPython39libsite-packagesgoogleoauth2service_account.py", line 221, in from_service_account_info
signer = _service_account_info.from_dict(
File "C:UsersjaremAppDataLocalProgramsPythonPython39libsite-packagesgoogleauth_service_account_info.py", line 58, in from_dict
signer = crypt.RSASigner.from_service_account_info(data)
File "C:UsersjaremAppDataLocalProgramsPythonPython39libsite-packagesgoogleauthcryptbase.py", line 113, in from_service_account_info
return cls.from_string(
File "C:UsersjaremAppDataLocalProgramsPythonPython39libsite-packagesgoogleauthcrypt_cryptography_rsa.py", line 133, in from_string
private_key = serialization.load_pem_private_key(
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagescryptographyhazmatprimitivesserializationbase.py", line 22, in load_pem_private_key
return ossl.load_pem_private_key(data, password)
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagescryptographyhazmatbackendsopensslbackend.py", line 921, in load_pem_private_key
return self._load_key(
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagescryptographyhazmatbackendsopensslbackend.py", line 1189, in _load_key
self._handle_key_loading_error()
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagescryptographyhazmatbackendsopensslbackend.py", line 1248, in _handle_key_loading_error
raise ValueError(
ValueError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=503841036, lib=60, reason=524556, reason_text=b'error:1E08010C:DECODER routines::unsupported')])
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0main.py", line 71, in <module>
cred = credentials.Certificate(my_credentials)
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagesfirebase_admincredentials.py", line 99, in __init__
raise ValueError('Failed to initialize a certificate credential. '
ValueError: Failed to initialize a certificate credential. Caused by: "('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=503841036, lib=60, reason=524556, reason_text=b'error:1E08010C:DECODER routines::unsupported')])"
All other environment variables seem to be readable. It might have something to do with the formatting of environment variables, but I don’t know what exactly. Nothing works and I don’t know the cause for this error. I work on Windows 11 and use cryptography version 35.0.0.
How do I fix this in order to extract the private key?
UPDATE:
What I have found is that when I print the private key directly, it is printed with new lines, e.g.:
-----BEGIN PRIVATE KEY-----
ZGVNvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDtuzZSrBuf4Lv3
DJTPJDj34jfknDJOJjfpitjrpZGVNvgIBADANBgkqhkiG9w0BAQZGVNvgI+DIome
ZGVNvgIBADANBgkqhkiG9w0BAQVdFkqJd5j53tZFgX5VLOf7g23/Zvgq+BFIHe34
(...MORE LINES HERE...)
-----END PRIVATE KEY-----
However, when I print the private key obtained by os.environ.get('PRIVATE KEY'), no newlines appear:
-----BEGIN PRIVATE KEY-----ZGVNvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDtuzZSrBuf4Lv3nDJTPJDj34jfknDJOJjfpitjrpZGVNvgIBADANBgkqhkiG9w0BAQZGVNvgI+DIomenZGVNvgIBADANBgkqhkiG9w0BAQVdFkqJd5j53tZFgX5VLOf7g23/Zvgq+BFIHe34n(...MORE CHARACTERS HERE...)-----END PRIVATE KEY-----
Seems there’s a problem with reading the "n" by the environment variable system. However, I still don’t know how to overcome this bug.
Answers:
I HAVE SOLVED THE PROBLEM:
To solve the problem with "n" I had to replace the raw string "n" with "n", as (presumably) the environment variables return a raw string, which treats backslash () as a literal character. The solution looks as follows:
my_credentials = {
"type": "service_account",
"project_id": "bookclub-b2db5",
"private_key_id": os.environ.get("PRIVATE_KEY_ID"),
"private_key": os.environ.get("PRIVATE_KEY").replace(r'n', 'n'), # CHANGE HERE
"client_email": os.environ.get("CLIENT_EMAIL"),
"client_id": os.environ.get("CLIENT_ID"),
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": os.environ.get("AUTH_PROVIDER_X509_CERT_URL"),
"client_x509_cert_url": os.environ.get("AUTH_PROVIDER_X509_CERT_URL")
}
Now, everything works.
This can be simplified to a single environment variable that contains the contents of the JSON file:
input_data = json.loads(os.environ.get("GCP_CREDENTIAL")
certificate = firebase_admin.credentials.Certificate(input_data)
firebase_admin.initialize_app(certificate)
I’m developing a Python web app with a Firestore realtime database using the firebase_admin library. The Firestore key comes in form of a .json file containing 10 variables. However, I want to store some of these variables as environment variables so they are not visible publicly. So, I don’t use a Firebase SDK .json file, but I create my own dictionary with the elements of this file. The dictionary looks like this and everything has been directly copied from the .json file:
my_credentials = {
"type": "service_account",
"project_id": "bookclub-b2db5",
"private_key_id": os.environ.get("PRIVATE_KEY_ID"),
"private_key": os.environ.get("PRIVATE_KEY"),
"client_email": os.environ.get("CLIENT_EMAIL"),
"client_id": os.environ.get("CLIENT_ID"),
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": os.environ.get("AUTH_PROVIDER_X509_CERT_URL"),
"client_x509_cert_url": os.environ.get("AUTH_PROVIDER_X509_CERT_URL")
}
I have set the private key as a PRIVATE_KEY environment variable.
The private key looks roughly like this (the characters here are made up):
"-----BEGIN PRIVATE KEY-----nEW8nYP9T840Sb8tQMinhZ(...MORE CHARACTERS HERE...)EW8nYP9T840Sb8tQMi/EW8nYP9T840Sb8tQMiEW8nYP9T840Sb8tQMi/EW8nYP9T840Sb8tQMi=n-----END PRIVATE KEY-----n"
Then, I try to create a Firebase Certificate from these credentials and initialize the application
cred = firebase_admin.credentials.Certificate(my_credentials)
firebase_admin.initialize_app(cred, {'databaseURL': 'https://myapp.firebasedatabase.app/'})
However, it turns out that the PRIVATE_KEY environment variable is not readable (cannot be deserialized), and it throws an error:
Traceback (most recent call last):
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagesfirebase_admincredentials.py", line 96, in __init__
self._g_credential = service_account.Credentials.from_service_account_info(
File "C:UsersjaremAppDataLocalProgramsPythonPython39libsite-packagesgoogleoauth2service_account.py", line 221, in from_service_account_info
signer = _service_account_info.from_dict(
File "C:UsersjaremAppDataLocalProgramsPythonPython39libsite-packagesgoogleauth_service_account_info.py", line 58, in from_dict
signer = crypt.RSASigner.from_service_account_info(data)
File "C:UsersjaremAppDataLocalProgramsPythonPython39libsite-packagesgoogleauthcryptbase.py", line 113, in from_service_account_info
return cls.from_string(
File "C:UsersjaremAppDataLocalProgramsPythonPython39libsite-packagesgoogleauthcrypt_cryptography_rsa.py", line 133, in from_string
private_key = serialization.load_pem_private_key(
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagescryptographyhazmatprimitivesserializationbase.py", line 22, in load_pem_private_key
return ossl.load_pem_private_key(data, password)
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagescryptographyhazmatbackendsopensslbackend.py", line 921, in load_pem_private_key
return self._load_key(
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagescryptographyhazmatbackendsopensslbackend.py", line 1189, in _load_key
self._handle_key_loading_error()
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagescryptographyhazmatbackendsopensslbackend.py", line 1248, in _handle_key_loading_error
raise ValueError(
ValueError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=503841036, lib=60, reason=524556, reason_text=b'error:1E08010C:DECODER routines::unsupported')])
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0main.py", line 71, in <module>
cred = credentials.Certificate(my_credentials)
File "C:UsersjaremDesktopdata_sciencepython-working-directory_MyAppsBookClub2.0venvlibsite-packagesfirebase_admincredentials.py", line 99, in __init__
raise ValueError('Failed to initialize a certificate credential. '
ValueError: Failed to initialize a certificate credential. Caused by: "('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=503841036, lib=60, reason=524556, reason_text=b'error:1E08010C:DECODER routines::unsupported')])"
All other environment variables seem to be readable. It might have something to do with the formatting of environment variables, but I don’t know what exactly. Nothing works and I don’t know the cause for this error. I work on Windows 11 and use cryptography version 35.0.0.
How do I fix this in order to extract the private key?
UPDATE:
What I have found is that when I print the private key directly, it is printed with new lines, e.g.:
-----BEGIN PRIVATE KEY-----
ZGVNvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDtuzZSrBuf4Lv3
DJTPJDj34jfknDJOJjfpitjrpZGVNvgIBADANBgkqhkiG9w0BAQZGVNvgI+DIome
ZGVNvgIBADANBgkqhkiG9w0BAQVdFkqJd5j53tZFgX5VLOf7g23/Zvgq+BFIHe34
(...MORE LINES HERE...)
-----END PRIVATE KEY-----
However, when I print the private key obtained by os.environ.get('PRIVATE KEY'), no newlines appear:
-----BEGIN PRIVATE KEY-----ZGVNvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDtuzZSrBuf4Lv3nDJTPJDj34jfknDJOJjfpitjrpZGVNvgIBADANBgkqhkiG9w0BAQZGVNvgI+DIomenZGVNvgIBADANBgkqhkiG9w0BAQVdFkqJd5j53tZFgX5VLOf7g23/Zvgq+BFIHe34n(...MORE CHARACTERS HERE...)-----END PRIVATE KEY-----
Seems there’s a problem with reading the "n" by the environment variable system. However, I still don’t know how to overcome this bug.
I HAVE SOLVED THE PROBLEM:
To solve the problem with "n" I had to replace the raw string "n" with "n", as (presumably) the environment variables return a raw string, which treats backslash () as a literal character. The solution looks as follows:
my_credentials = {
"type": "service_account",
"project_id": "bookclub-b2db5",
"private_key_id": os.environ.get("PRIVATE_KEY_ID"),
"private_key": os.environ.get("PRIVATE_KEY").replace(r'n', 'n'), # CHANGE HERE
"client_email": os.environ.get("CLIENT_EMAIL"),
"client_id": os.environ.get("CLIENT_ID"),
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": os.environ.get("AUTH_PROVIDER_X509_CERT_URL"),
"client_x509_cert_url": os.environ.get("AUTH_PROVIDER_X509_CERT_URL")
}
Now, everything works.
This can be simplified to a single environment variable that contains the contents of the JSON file:
input_data = json.loads(os.environ.get("GCP_CREDENTIAL")
certificate = firebase_admin.credentials.Certificate(input_data)
firebase_admin.initialize_app(certificate)