Could not deserialize key data on decoding JWT python
Question:
I am using pyjwt library for decoding the JWT token. I got this error when I am decoding. The code was given in the documantation.
import jwt
encoded_jwt='''eyJ0eXAiOiJKV1QiLCJhbG......'''
secret=b''''-----BEGIN PUBLIC KEY-----
MIIFRjCCBC6gAwIBAgIQCIdSGhpikQCjOIY154XoqzANBgkqhkiG9w0BAQsFADBN
......
-----END PUBLIC KEY-----'''
print(jwt.decode(encoded_jwt, secret , algorithms=['RS256']))
raise ValueError(“Could not deserialize key data.”) ValueError: Could
not deserialize key data.
Could You please help me in resolving it beacuse when I use this it in the JWT website it’s working.
This is the full error log..
Traceback (most recent call last): File
“/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/jwt/algorithms.py”,
line 205, in prepare_key
key = load_pem_private_key(key, password=None, backend=default_backend()) File
“/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/primitives/serialization.py”,
line 20, in load_pem_private_key
return backend.load_pem_private_key(data, password) File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py”,
line 1014, in load_pem_private_key
password, File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py”,
line 1233, in _load_key
self._handle_key_loading_error() File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py”,
line 1291, in _handle_key_loading_error
raise ValueError(“Could not deserialize key data.”) ValueError: Could not deserialize key data.
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File
“/home/sathiyakugan/PycharmProjects/JWTsample/sample.py”, line 45, in
print(jwt.decode(encoded_jwt, secret , algorithms=[‘RS256’])) File
“/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/jwt/api_jwt.py”,
line 93, in decode
jwt, key=key, algorithms=algorithms, options=options, **kwargs File
“/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/jwt/api_jws.py”,
line 157, in decode
key, algorithms) File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/jwt/api_jws.py”,
line 221, in _verify_signature
key = alg_obj.prepare_key(key) File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/jwt/algorithms.py”,
line 207, in prepare_key
key = load_pem_public_key(key, backend=default_backend()) File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/primitives/serialization.py”,
line 24, in load_pem_public_key
return backend.load_pem_public_key(data) File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py”,
line 1040, in load_pem_public_key
self._handle_key_loading_error() File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py”,
line 1291, in _handle_key_loading_error
raise ValueError(“Could not deserialize key data.”) ValueError: Could not deserialize key data.
Process finished with exit code 1
Answers:
How did you encode your jwt? Use one of the approaches below
Encoding & Decoding Tokens with RS256 (RSA)
encoded = jwt.encode({'some': 'payload'}, private_key, algorithm='RS256')
decoded = jwt.decode(encoded, public_key, algorithms='RS256')
Reading the Claimset without Validation
jwt.decode(encoded, verify=False)
{u'some': u'payload'}
Or use same secret to encode and decode the jwt, one of the approach should work. In my case I used jwt.decode(token, verify=False)
because my server has already did the signature validation for me, I only need to get the claimset.
There are some issues in the pyjwt library. and you must get the public key from the certificate.
I used openssl x509 -pubkey -noout -in cert.pem > pubkey.pem
then from the public key I could easily decode it using authlib library.
from authlib.specs.rfc7519 import jwt
encoded_jwt='''eyJ0eXAiOiJ....'''
secret=b'''-----BEGIN PUBLIC KEY-----
......
-----END PUBLIC KEY-----'''
claims = jwt.decode(encoded_jwt, secret)
print(claims)
Use the authlib library, I never managed to decode keycloak tokens with pyjwt
.
You need a public_key
, I assume you have it.
from authlib.jose import jwt
key = '-----BEGIN PUBLIC KEY-----n' + public_key + 'n-----END PUBLIC KEY-----'
key_binary = key.encode('ascii')
try:
claims = jwt.decode(encoded,key_binary)
claims.validate()
#do some logic here
#...
ProTip: you may grab the public key easily from your auth server (in my case Keycloak) at some endpoint:
url = 'http://localhost:8080/auth/realms/your_realm'
with urllib.request.urlopen(url) as r:
response = r.read()
public_key = json.loads(response)['public_key']
It’s a good idea to use your RSA keys with OpenSSL:
openssl genrsa -out jwt-key 4096
openssl rsa -in jwt-key -pubout > jwt-key.pub
Reference: link
Updated Text:
It seems you’re encountering a key deserialization problem with the PyJWT library. This issue often arises from an incompatible or incorrectly formatted key. To resolve this, I suggest generating a new RSA key pair using OpenSSL. Here’s a step-by-step explanation:
-
Generating the RSA Private Key:
- Command:
openssl genrsa -out jwt-key 4096
- This command uses OpenSSL to create an RSA private key.
genrsa
generates the RSA key.
-out jwt-key
defines the output filename for the private key.
4096
is the key size, offering a balance between security and performance.
-
Extracting the Public Key:
- Command:
openssl rsa -in jwt-key -pubout > jwt-key.pub
- This command extracts the public key from the generated private key.
-in jwt-key
specifies the input file, the private key from the first step.
-pubout
signals the extraction of the public key.
jwt-key.pub
is where the public key will be saved.
By using these commands, you’ll generate a new RSA key pair. The private key (jwt-key
) is for signing the JWT, and the public key (jwt-key.pub
) is for verifying the signature. Remember to securely store the private key and only share the public key.
After generating these keys, update your Python code to use the new keys, ensuring the public key is read in the format expected by the PyJWT library. This approach should solve the deserialization issue with your current key.
If you get this error double check your public key is exactly right, new lines are important.
key = '''-----BEGIN PUBLIC KEY-----
<main key here>
-----END PUBLIC KEY-----'''
For me, there were spaces in the key (silly IDE) which caused this issue, make sure that’s not the case with you
MacOS Catalina 10.15.4 Python 2.7.16.
This solved the problem in my case
pip install cryptography==2.3
save the public key into the file and pass the path and use that.
- public_key = open(path).read()
- payload = jwt.decode(token,
public_key, algorithms=[‘RS256’])
If the problem was the fact that you are using certificate you can just extract public / private key from the certificate:
from cryptography.x509 import load_pem_x509_certificate
cert_str = b"-----BEGIN CERTIFICATE-----MIIDETCCAfm..."
cert_obj = load_pem_x509_certificate(cert_str)
public_key = cert_obj.public_key()
private_key = cert_obj.private_key()
For me what worked was setting the jwt verify signature to false:
import jwt
token = st_javascript('''localStorage.getItem("some_token")''') # fetch your token of interest
decoded = jwt.decode(token, options={"verify_signature": False}) # decode the token
Note: This bypasses JWT verification and should only be used if you don’t care wether the JWT is cryptographically invalid or valid.
For more info you can view the jwt documentation.
For pyjwt 2.8.0
, it worked for me when using options={"verify_signature": False}
instead of verify=False
jwt.decode(token.encode(), options={"verify_signature": False})
If you have the JWKS url of your Authorization Server. Reference Use dynamic keys
from authlib.jose import jwt
response = requests.get('http://localhost:9090/oauth2/jwks')
response = response.json()
claims = jwt.decode(encoded, response)
print("claims validate:", claims.validate())
print("claims:", claims)
I tried manually passing the Public key to jwt.decode()
and with pyjwt as well.
I am using pyjwt library for decoding the JWT token. I got this error when I am decoding. The code was given in the documantation.
import jwt
encoded_jwt='''eyJ0eXAiOiJKV1QiLCJhbG......'''
secret=b''''-----BEGIN PUBLIC KEY-----
MIIFRjCCBC6gAwIBAgIQCIdSGhpikQCjOIY154XoqzANBgkqhkiG9w0BAQsFADBN
......
-----END PUBLIC KEY-----'''
print(jwt.decode(encoded_jwt, secret , algorithms=['RS256']))
raise ValueError(“Could not deserialize key data.”) ValueError: Could
not deserialize key data.
Could You please help me in resolving it beacuse when I use this it in the JWT website it’s working.
This is the full error log..
Traceback (most recent call last): File
“/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/jwt/algorithms.py”,
line 205, in prepare_key
key = load_pem_private_key(key, password=None, backend=default_backend()) File
“/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/primitives/serialization.py”,
line 20, in load_pem_private_key
return backend.load_pem_private_key(data, password) File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py”,
line 1014, in load_pem_private_key
password, File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py”,
line 1233, in _load_key
self._handle_key_loading_error() File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py”,
line 1291, in _handle_key_loading_error
raise ValueError(“Could not deserialize key data.”) ValueError: Could not deserialize key data.During handling of the above exception, another exception occurred:
Traceback (most recent call last): File
“/home/sathiyakugan/PycharmProjects/JWTsample/sample.py”, line 45, inprint(jwt.decode(encoded_jwt, secret , algorithms=[‘RS256’])) File
“/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/jwt/api_jwt.py”,
line 93, in decode
jwt, key=key, algorithms=algorithms, options=options, **kwargs File
“/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/jwt/api_jws.py”,
line 157, in decode
key, algorithms) File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/jwt/api_jws.py”,
line 221, in _verify_signature
key = alg_obj.prepare_key(key) File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/jwt/algorithms.py”,
line 207, in prepare_key
key = load_pem_public_key(key, backend=default_backend()) File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/primitives/serialization.py”,
line 24, in load_pem_public_key
return backend.load_pem_public_key(data) File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py”,
line 1040, in load_pem_public_key
self._handle_key_loading_error() File “/home/sathiyakugan/PycharmProjects/Python/venv/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py”,
line 1291, in _handle_key_loading_error
raise ValueError(“Could not deserialize key data.”) ValueError: Could not deserialize key data.Process finished with exit code 1
How did you encode your jwt? Use one of the approaches below
Encoding & Decoding Tokens with RS256 (RSA)
encoded = jwt.encode({'some': 'payload'}, private_key, algorithm='RS256')
decoded = jwt.decode(encoded, public_key, algorithms='RS256')
Reading the Claimset without Validation
jwt.decode(encoded, verify=False)
{u'some': u'payload'}
Or use same secret to encode and decode the jwt, one of the approach should work. In my case I used jwt.decode(token, verify=False)
because my server has already did the signature validation for me, I only need to get the claimset.
There are some issues in the pyjwt library. and you must get the public key from the certificate.
I used openssl x509 -pubkey -noout -in cert.pem > pubkey.pem
then from the public key I could easily decode it using authlib library.
from authlib.specs.rfc7519 import jwt
encoded_jwt='''eyJ0eXAiOiJ....'''
secret=b'''-----BEGIN PUBLIC KEY-----
......
-----END PUBLIC KEY-----'''
claims = jwt.decode(encoded_jwt, secret)
print(claims)
Use the authlib library, I never managed to decode keycloak tokens with pyjwt
.
You need a public_key
, I assume you have it.
from authlib.jose import jwt
key = '-----BEGIN PUBLIC KEY-----n' + public_key + 'n-----END PUBLIC KEY-----'
key_binary = key.encode('ascii')
try:
claims = jwt.decode(encoded,key_binary)
claims.validate()
#do some logic here
#...
ProTip: you may grab the public key easily from your auth server (in my case Keycloak) at some endpoint:
url = 'http://localhost:8080/auth/realms/your_realm'
with urllib.request.urlopen(url) as r:
response = r.read()
public_key = json.loads(response)['public_key']
It’s a good idea to use your RSA keys with OpenSSL:
openssl genrsa -out jwt-key 4096
openssl rsa -in jwt-key -pubout > jwt-key.pub
Reference: link
Updated Text:
It seems you’re encountering a key deserialization problem with the PyJWT library. This issue often arises from an incompatible or incorrectly formatted key. To resolve this, I suggest generating a new RSA key pair using OpenSSL. Here’s a step-by-step explanation:
-
Generating the RSA Private Key:
- Command:
openssl genrsa -out jwt-key 4096
- This command uses OpenSSL to create an RSA private key.
genrsa
generates the RSA key.-out jwt-key
defines the output filename for the private key.4096
is the key size, offering a balance between security and performance.
- Command:
-
Extracting the Public Key:
- Command:
openssl rsa -in jwt-key -pubout > jwt-key.pub
- This command extracts the public key from the generated private key.
-in jwt-key
specifies the input file, the private key from the first step.-pubout
signals the extraction of the public key.jwt-key.pub
is where the public key will be saved.
- Command:
By using these commands, you’ll generate a new RSA key pair. The private key (jwt-key
) is for signing the JWT, and the public key (jwt-key.pub
) is for verifying the signature. Remember to securely store the private key and only share the public key.
After generating these keys, update your Python code to use the new keys, ensuring the public key is read in the format expected by the PyJWT library. This approach should solve the deserialization issue with your current key.
If you get this error double check your public key is exactly right, new lines are important.
key = '''-----BEGIN PUBLIC KEY-----
<main key here>
-----END PUBLIC KEY-----'''
For me, there were spaces in the key (silly IDE) which caused this issue, make sure that’s not the case with you
MacOS Catalina 10.15.4 Python 2.7.16.
This solved the problem in my case
pip install cryptography==2.3
save the public key into the file and pass the path and use that.
- public_key = open(path).read()
- payload = jwt.decode(token,
public_key, algorithms=[‘RS256’])
If the problem was the fact that you are using certificate you can just extract public / private key from the certificate:
from cryptography.x509 import load_pem_x509_certificate
cert_str = b"-----BEGIN CERTIFICATE-----MIIDETCCAfm..."
cert_obj = load_pem_x509_certificate(cert_str)
public_key = cert_obj.public_key()
private_key = cert_obj.private_key()
For me what worked was setting the jwt verify signature to false:
import jwt
token = st_javascript('''localStorage.getItem("some_token")''') # fetch your token of interest
decoded = jwt.decode(token, options={"verify_signature": False}) # decode the token
Note: This bypasses JWT verification and should only be used if you don’t care wether the JWT is cryptographically invalid or valid.
For more info you can view the jwt documentation.
For pyjwt 2.8.0
, it worked for me when using options={"verify_signature": False}
instead of verify=False
jwt.decode(token.encode(), options={"verify_signature": False})
If you have the JWKS url of your Authorization Server. Reference Use dynamic keys
from authlib.jose import jwt
response = requests.get('http://localhost:9090/oauth2/jwks')
response = response.json()
claims = jwt.decode(encoded, response)
print("claims validate:", claims.validate())
print("claims:", claims)
I tried manually passing the Public key to jwt.decode()
and with pyjwt as well.