google-cloud-dataproc

How to list Dataproc operations in `google-cloud-dataproc` client Question: I am looking for a way to do something similar to CLI’s gcloud dataproc operations list –filter "…". The minimal code example: from google.cloud import dataproc_v1 region = ‘us-west1’ client_options = {"api_endpoint": f"{region}-dataproc.googleapis.com:443"} dataproc_cluster_client = dataproc_v1.ClusterControllerClient(client_options=client_options) def list_operations(dataproc_cluster_client, region): for op in dataproc_cluster_client.list_operations( request={"filter": f"operationType = …

Dataproc: Errors when reading and writing data from BigQuery using PySpark Question: I am trying to read some BigQuery data, (ID: my-project.mydatabase.mytable [original names protected]) from a user-managed Jupyter Notebook instance, inside Dataproc Workbench. What I am trying is inspired in this, and more specifically, the code is (please read some additional comments, on the …