Connect CloudSQL Postgres Db using Python Question: So I am DevOps engineer please excuse for my lack of knowledge in python development. I am trying to connect a CloudSQL Postgres 14 Db using Python. So that I can insert data and read data from my application. Can someone help me understand why I am getting …
How to bypass the TTL of 30 seconds while waiting response of post request from external server Question: I am building a view in Django , which will send a POST request to chat-gpt API, the problem that I am facing that the response from chat-gpt is taking more than 30 seconds (we are having …
Is It possible to trigger a Dag from an on_failure_callback? Question: I would like to trigger a Dag when a task is failed. I want to use the "on_failure_callback", however, I have not found information about it. Do you know if it is possible to trigger the dag from the "on_failure_callback"? In the past, I …
Dataflow Template launch failure on pickling Question: My Dataflow pipeline is as followed pipeline_options = PipelineOptions( pipeline_args, streaming=True, save_main_session=True, sdk_location="container" ) with Pipeline(options=pipeline_options) as pipeline: ( pipeline | f"Read event topic" >> io.ReadFromPubSub(topic=input_topic).with_output_types(bytes) | "Convert to string" >> beam.Map(lambda msg: msg.decode("utf=8")) | f"Transform event" >> beam.Map(transform_message, event_name=event_name) | f"Write to output topic" >> beam.Map(publish_to_output_topic) ) …
Can't write to file in Google Cloud Question: Currently, I have an SQL Database that I wish to edit locally. This is for a flask app made in python 3, using sqlite3. con = sqlite3.connect(‘Database/accounts.db’) cur = con.cursor() cur.execute(f"INSERT INTO Requests (Type, Argument, Priority, Time) VALUES (‘{a}’,'{b}’,{c},'{d}’)") con.commit() Except I keep getting the error: attempt …
Unable to stream API response in Flask application on Google Cloud application Question: I’m developing a little testing website using the OpenAI API. I’m trying to stream GPT’s response, just like how it’s done on https://chat.openai.com/chat. This works just fine when running my Flask application on a local development server, but when I deploy this …
How to list Dataproc operations in `google-cloud-dataproc` client Question: I am looking for a way to do something similar to CLI’s gcloud dataproc operations list –filter "…". The minimal code example: from google.cloud import dataproc_v1 region = ‘us-west1’ client_options = {"api_endpoint": f"{region}-dataproc.googleapis.com:443"} dataproc_cluster_client = dataproc_v1.ClusterControllerClient(client_options=client_options) def list_operations(dataproc_cluster_client, region): for op in dataproc_cluster_client.list_operations( request={"filter": f"operationType = …
Signed Url is working after the expiration date Cloud Storage python Question: I have found this question that seems dead so i will add more context. i am using the python google-cloud-storage sdk to generate signed urls using blob.generated_signed_url(). Here is the full method i use : blob.generate_signed_url(expiration=datetime.now() + timedelta(minutes=1), version="v4") I live in France …
Deep Learning training slower on Google Cloud VM than Local PC Question: I am trying to train an LSTM neural network using Pytorch. On my own computer the process is quite slow due to the complexity of the model and size of the dataset. My initial thought was to move the training to a cloud …
Where to securely store Google_Application_Credentials in Django project? Question: I have a Django project that uses the Google API. I created a docker-compose.yml which has my Django container and Nginx container. It builds successfully but when I run docker-compose up, I get following error: google.auth.exceptions.DefaultCredentialsError: File /file/path/credentials.json was not found. I have installed the Google …
