How to copy entire structure between storage accounts in python Question: my case is the following: Two Azure Storage Accounts (Source/Destination) Source Account may contain multiple containers, folders, blobs, etc. All of the above needs to be copied exactly in the same structure to the DESTINATION account. If any elements already exist in the Destination …
Write a variable's data as it is in ADLS file Question: I want to write the content of a variable that is created dynamically in the program to a ADLS file. This is how I am getting the data – @dataclass class pipeline_run: id:str group_id:str run_start:str run_end:str pipeline_name:str pipeline_status:str parameters:str message:str addl_properties:str runs = adf_client.pipeline_runs.query_by_factory(rg_name, …
pyspark createDataframe typeerror: structtype can not accept object 'id' in type <class 'str'> Question: An API call is returning DICT type response similar to the output below: {‘Account’: {‘id’: 123, ‘externalIdentifier’: None, ‘name’: ‘test acct’, ‘accountNumber’: None, ‘Rep’: None, ‘organizationId’: 123, ‘streetAddress’: ‘123 Main Road’, ‘streetAddressCity’: ‘Town City’, ‘streetAddressState’: ‘Texas’, ‘streetAddressZipCode’: ‘76123’, ‘contact’: [{‘id’: 10001, …
Calling referenced functions after mssparkutil.notebook.run? Question: How can I call functions defined in a different Synapse notebook after running the notebook with mssparkutils.notebook.run()? example: #parameters value = "test" from notebookutils import mssparkutils mssparkutils.notebook.run("function definitions", 60, {"param": value}) df = load_cosmos_data() #defined in ‘function definitions’ notebook This fails with: NameError: name ‘load_cosmos_data’ is not defined I …
M2M Client Credential Flow between NetSuite and Synapse Question: I am looking to create a flow somewhere in the Azure stack to allow me to get M2M authentication between Azure Synapse and NetSuite. The goal is to be able to drop the use of the ODBC connector and switch to making REST calls for pulling …
