How to connect to Microsoft Dataverse API Endpoint with Python
Question:
I’m facing an issue in connecting to Microsoft Dataverse API endpoint through Python.
The code is broken down into three parts:
A) Token Generation
B) Endpoint API call Option 1
C) Endpoint API call Option 2
################ Token generation code begins ################
import adal
# Tenant ID for your Azure Subscription
TENANT_ID = 'abc'
# Your Service Principal App ID
CLIENT = '52'
# Your Service Principal Password
KEY = '43'
# subscription_id = 'xxxxxxx'
authority_url = 'https://login.microsoftonline.com/'+TENANT_ID
context = adal.AuthenticationContext(authority_url)
token = context.acquire_token_with_client_credentials(
resource='https://orgabc.api.crm8.dynamics.com/',
client_id=CLIENT,
client_secret=KEY
)
print('Fresh token is: ', token["accessToken"])
print('Full response is: ', token)
################ Token generation code ends ################
################ Method/Option 1: Token generation code begins ################
import requests
import json
import msal
config = {
"authority": "https://login.microsoftonline.com/abc",
"client_id": "52",
"client_secret": "43",
"scope": ["https://orgabc.crm8.dynamics.com/.default"],
}
app = msal.ConfidentialClientApplication(
config["client_id"],
authority=config["authority"],
client_credential=config["client_secret"])
result = app.acquire_token_silent(config["scope"], account=None)
if not result:
result = app.acquire_token_for_client(scopes=config["scope"])
# bearerToken = result['access_token']
bearerToken = token["accessToken"]
url = "https://orgabc.api.crm8.dynamics.com/api/data/v9.2"
headers = {
"Accept": "application/json",
"Content-type": "application/json",
"Authorization": "Bearer "+bearerToken,
}
response = requests.request("GET", url, headers = headers)
print(response)
################ Method/Option 1: Token generation code ends ################
# ################ Method/Option 2: Token generation code begins ################
# import requests
# import json
#
# # set these values to retrieve the oauth token
# crmorg = 'https://orgabc.crm8.dynamics.com' # base url for crm org
# clientid = '52' # application client id
# username = '[email protected]' # username
# userpassword = 'HgN@6013' # password
# tokenendpoint = 'https://login.microsoftonline.com/abc/oauth2/v2.0/token' # oauth token endpoint
#
# # set these values to query your crm data
# crmwebapi = 'https://orgabc.crm8.dynamics.com/api/data/v9.2' # full path to web api endpoint
# # crmwebapiquery = '/contacts?$select=fullname,contactid' # web api query (include leading /)
# crmwebapiquery = '/accounts' # web api query (include leading /)
#
#
# # build the authorization token request
# tokenpost = {
# 'client_id': clientid,
# 'resource': crmorg,
# 'username': username,
# 'password': userpassword,
# 'grant_type': 'password'
# }
#
# # make the token request
# tokenres = requests.post(tokenendpoint, data=tokenpost)
#
# # set accesstoken variable to empty string
# accesstoken = ''
#
# # extract the access token
# try:
# accesstoken = token["accessToken"]
# except(KeyError):
# # handle any missing key errors
# print('Could not get access token')
#
# # if we have an accesstoken
# if (accesstoken != ''):
# # prepare the crm request headers
# crmrequestheaders = {
# 'Authorization': 'Bearer ' + token["accessToken"],
# 'OData-MaxVersion': '4.0',
# 'OData-Version': '4.0',
# 'Accept': 'application/json',
# 'Content-Type': 'application/json; charset=utf-8',
# 'Prefer': 'odata.maxpagesize=500',
# 'Prefer': 'odata.include-annotations=OData.Community.Display.V1.FormattedValue'
# }
#
# # make the crm request
# crmres = requests.get(crmwebapi + crmwebapiquery, headers=crmrequestheaders)
#
# try:
# # get the response json
# crmresults = crmres.json()
#
# # loop through it
# for x in crmresults['value']:
# print(x['fullname'] + ' - ' + x['contactid'])
# except KeyError:
# # handle any missing key errors
# print('Could not parse CRM results')
# ################ Method/Option 2: Token generation code ends ################
The block 1 of the code is working fine, where we’re generating the token, but while we try block 2 or 3, in both the cases the code fails with the below error (full traceback):
Traceback (most recent call last):
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3connectionpool.py", line 597, in urlopen
httplib_response = self._make_request(conn, method, url,
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3connectionpool.py", line 849, in _validate_conn
conn.connect()
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3connection.py", line 349, in connect
self.sock = ssl_wrap_socket(
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3utilssl_.py", line 359, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:Usersaakashbasu1AppDataLocalProgramsPythonPython39libssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "C:Usersaakashbasu1AppDataLocalProgramsPythonPython39libssl.py", line 1040, in _create
self.do_handshake()
File "C:Usersaakashbasu1AppDataLocalProgramsPythonPython39libssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1129)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesrequestsadapters.py", line 440, in send
resp = conn.urlopen(
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3connectionpool.py", line 637, in urlopen
retries = retries.increment(method, url, error=e, _pool=self,
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3utilretry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='orgabc.api.crm8.dynamics.com', port=443): Max retries exceeded with url: /api/data/v9.2 (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1129)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDinternal_experimentsazure_ad_token_generator.py", line 60, in <module>
response = requests.request("GET", url, headers = headers)
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesrequestsapi.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesrequestssessions.py", line 529, in request
resp = self.send(prep, **send_kwargs)
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesrequestssessions.py", line 645, in send
r = adapter.send(request, **kwargs)
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesrequestsadapters.py", line 517, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='orgabc.api.crm8.dynamics.com', port=443): Max retries exceeded with url: /api/data/v9.2 (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1129)')))
There’s not much help available on the open forum for this problem for Dataverse. There is a C# code available which works fine, but we need to use Python for the same. C# code for the same is:
using Microsoft.AspNetCore.Http;
using Microsoft.Graph;
using Microsoft.WindowsAzure.ActiveDirectory.Authentication;
using MySql.Data.MySqlClient.Memcached;
using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
namespace AccessToken
{
class Program
{
static void Main(string[] args)
{
string serviceUrl = "https://orgabc.crm8.dynamics.com";
string clientId = "52";//"<your app id>";
string secret = "41";//"<your app secret>";
AuthenticationContext authContext = new AuthenticationContext("https://login.microsoftonline.com/53");//tenant ID
ClientCredential credential = new ClientCredential(clientId, secret);
AuthenticationResult result = authContext.AcquireToken(serviceUrl,credential);
string accessToken = result.AccessToken;
Console.WriteLine("Hello World!");
Console.WriteLine("Acces Token for given :"+accessToken);
//Retrieving data here
using (var client = new HttpClient())
{
var tokenType = "Bearer"; //Other APIs use Bearer or other auth types.
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(tokenType, accessToken);//token type=Bearer, accesstoken refers to token which we generated.
//https://orgabc.crm8.dynamics.com refers to org URL
var content = client.GetStringAsync("https://orgabc.crm8.dynamics.com/api/data/v9.2/accounts");//accounts refers to the User Table Logical Name.
Console.WriteLine("Acces Token for given :" + content.Result);
}
//end of retrieve data.
}
}
}
What we tried?
- Changing the API version to 9.1 and 9.0 from the actual 9.2 as per microsoft documentation
- Lowering urllib version as per suggestions from multiple SO Q/A like this
- Working on the same connection details with Postman Tool works perfectly fine
So, narrowing down on it seems the issue is with the code hitting the endpoint. Any help?
Answers:
The code has no bug, it is correctly written. The issue was with the local machine’s firewall/proxy/VPN setting issues which was blocking the actual API call. It works fine when run on an open system.
I am trying to import all my dataverse tables in python using this code but getting a 401 response error. I have been trying option 1. I am able to generate token but not able to get a response.
I’m facing an issue in connecting to Microsoft Dataverse API endpoint through Python.
The code is broken down into three parts:
A) Token Generation
B) Endpoint API call Option 1
C) Endpoint API call Option 2
################ Token generation code begins ################
import adal
# Tenant ID for your Azure Subscription
TENANT_ID = 'abc'
# Your Service Principal App ID
CLIENT = '52'
# Your Service Principal Password
KEY = '43'
# subscription_id = 'xxxxxxx'
authority_url = 'https://login.microsoftonline.com/'+TENANT_ID
context = adal.AuthenticationContext(authority_url)
token = context.acquire_token_with_client_credentials(
resource='https://orgabc.api.crm8.dynamics.com/',
client_id=CLIENT,
client_secret=KEY
)
print('Fresh token is: ', token["accessToken"])
print('Full response is: ', token)
################ Token generation code ends ################
################ Method/Option 1: Token generation code begins ################
import requests
import json
import msal
config = {
"authority": "https://login.microsoftonline.com/abc",
"client_id": "52",
"client_secret": "43",
"scope": ["https://orgabc.crm8.dynamics.com/.default"],
}
app = msal.ConfidentialClientApplication(
config["client_id"],
authority=config["authority"],
client_credential=config["client_secret"])
result = app.acquire_token_silent(config["scope"], account=None)
if not result:
result = app.acquire_token_for_client(scopes=config["scope"])
# bearerToken = result['access_token']
bearerToken = token["accessToken"]
url = "https://orgabc.api.crm8.dynamics.com/api/data/v9.2"
headers = {
"Accept": "application/json",
"Content-type": "application/json",
"Authorization": "Bearer "+bearerToken,
}
response = requests.request("GET", url, headers = headers)
print(response)
################ Method/Option 1: Token generation code ends ################
# ################ Method/Option 2: Token generation code begins ################
# import requests
# import json
#
# # set these values to retrieve the oauth token
# crmorg = 'https://orgabc.crm8.dynamics.com' # base url for crm org
# clientid = '52' # application client id
# username = '[email protected]' # username
# userpassword = 'HgN@6013' # password
# tokenendpoint = 'https://login.microsoftonline.com/abc/oauth2/v2.0/token' # oauth token endpoint
#
# # set these values to query your crm data
# crmwebapi = 'https://orgabc.crm8.dynamics.com/api/data/v9.2' # full path to web api endpoint
# # crmwebapiquery = '/contacts?$select=fullname,contactid' # web api query (include leading /)
# crmwebapiquery = '/accounts' # web api query (include leading /)
#
#
# # build the authorization token request
# tokenpost = {
# 'client_id': clientid,
# 'resource': crmorg,
# 'username': username,
# 'password': userpassword,
# 'grant_type': 'password'
# }
#
# # make the token request
# tokenres = requests.post(tokenendpoint, data=tokenpost)
#
# # set accesstoken variable to empty string
# accesstoken = ''
#
# # extract the access token
# try:
# accesstoken = token["accessToken"]
# except(KeyError):
# # handle any missing key errors
# print('Could not get access token')
#
# # if we have an accesstoken
# if (accesstoken != ''):
# # prepare the crm request headers
# crmrequestheaders = {
# 'Authorization': 'Bearer ' + token["accessToken"],
# 'OData-MaxVersion': '4.0',
# 'OData-Version': '4.0',
# 'Accept': 'application/json',
# 'Content-Type': 'application/json; charset=utf-8',
# 'Prefer': 'odata.maxpagesize=500',
# 'Prefer': 'odata.include-annotations=OData.Community.Display.V1.FormattedValue'
# }
#
# # make the crm request
# crmres = requests.get(crmwebapi + crmwebapiquery, headers=crmrequestheaders)
#
# try:
# # get the response json
# crmresults = crmres.json()
#
# # loop through it
# for x in crmresults['value']:
# print(x['fullname'] + ' - ' + x['contactid'])
# except KeyError:
# # handle any missing key errors
# print('Could not parse CRM results')
# ################ Method/Option 2: Token generation code ends ################
The block 1 of the code is working fine, where we’re generating the token, but while we try block 2 or 3, in both the cases the code fails with the below error (full traceback):
Traceback (most recent call last):
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3connectionpool.py", line 597, in urlopen
httplib_response = self._make_request(conn, method, url,
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3connectionpool.py", line 849, in _validate_conn
conn.connect()
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3connection.py", line 349, in connect
self.sock = ssl_wrap_socket(
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3utilssl_.py", line 359, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:Usersaakashbasu1AppDataLocalProgramsPythonPython39libssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "C:Usersaakashbasu1AppDataLocalProgramsPythonPython39libssl.py", line 1040, in _create
self.do_handshake()
File "C:Usersaakashbasu1AppDataLocalProgramsPythonPython39libssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1129)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesrequestsadapters.py", line 440, in send
resp = conn.urlopen(
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3connectionpool.py", line 637, in urlopen
retries = retries.increment(method, url, error=e, _pool=self,
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesurllib3utilretry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='orgabc.api.crm8.dynamics.com', port=443): Max retries exceeded with url: /api/data/v9.2 (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1129)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDinternal_experimentsazure_ad_token_generator.py", line 60, in <module>
response = requests.request("GET", url, headers = headers)
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesrequestsapi.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesrequestssessions.py", line 529, in request
resp = self.send(prep, **send_kwargs)
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesrequestssessions.py", line 645, in send
r = adapter.send(request, **kwargs)
File "C:Usersaakashbasu1PycharmProjectsPersonal_RnDvenvlibsite-packagesrequestsadapters.py", line 517, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='orgabc.api.crm8.dynamics.com', port=443): Max retries exceeded with url: /api/data/v9.2 (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1129)')))
There’s not much help available on the open forum for this problem for Dataverse. There is a C# code available which works fine, but we need to use Python for the same. C# code for the same is:
using Microsoft.AspNetCore.Http;
using Microsoft.Graph;
using Microsoft.WindowsAzure.ActiveDirectory.Authentication;
using MySql.Data.MySqlClient.Memcached;
using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
namespace AccessToken
{
class Program
{
static void Main(string[] args)
{
string serviceUrl = "https://orgabc.crm8.dynamics.com";
string clientId = "52";//"<your app id>";
string secret = "41";//"<your app secret>";
AuthenticationContext authContext = new AuthenticationContext("https://login.microsoftonline.com/53");//tenant ID
ClientCredential credential = new ClientCredential(clientId, secret);
AuthenticationResult result = authContext.AcquireToken(serviceUrl,credential);
string accessToken = result.AccessToken;
Console.WriteLine("Hello World!");
Console.WriteLine("Acces Token for given :"+accessToken);
//Retrieving data here
using (var client = new HttpClient())
{
var tokenType = "Bearer"; //Other APIs use Bearer or other auth types.
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(tokenType, accessToken);//token type=Bearer, accesstoken refers to token which we generated.
//https://orgabc.crm8.dynamics.com refers to org URL
var content = client.GetStringAsync("https://orgabc.crm8.dynamics.com/api/data/v9.2/accounts");//accounts refers to the User Table Logical Name.
Console.WriteLine("Acces Token for given :" + content.Result);
}
//end of retrieve data.
}
}
}
What we tried?
- Changing the API version to 9.1 and 9.0 from the actual 9.2 as per microsoft documentation
- Lowering urllib version as per suggestions from multiple SO Q/A like this
- Working on the same connection details with Postman Tool works perfectly fine
So, narrowing down on it seems the issue is with the code hitting the endpoint. Any help?
The code has no bug, it is correctly written. The issue was with the local machine’s firewall/proxy/VPN setting issues which was blocking the actual API call. It works fine when run on an open system.
I am trying to import all my dataverse tables in python using this code but getting a 401 response error. I have been trying option 1. I am able to generate token but not able to get a response.