Django google kubernetes client not running exe inside the job Question: I have a docker image that I want to run inside my django code. Inside that image there is an executable that I have written using c++ that writes it’s output to google cloud storage. Normally when I run the django code like this: …
Upload zip file from Google cloud storage to SFTP server without loading into memory using Python Question: I want to upload large zip files from google cloud bucket to SFTP server without loading into memory and cloud function. I am using pysftp for SFTP transfer. with pysftp.Connection(host="SFTP Server", username=’test’, password=’test’, cnopts=cnopts) as sftp: sftp.cwd("/tmp") sftp.put("zip_file_from_google_bucket") …
Google BigQuery query in Python works when using result(), but Permission issue when using to_dataframe() Question: I’ve run into a problem after upgrades of my pip packages and my bigquery connector that returns query results suddenly stopped working with following error message from google.cloud import bigquery from google.oauth2 import service_account credentials = service_account.Credentials.from_service_account_file(‘path/to/file’, scopes=[‘https://www.googleapis.com/auth/cloud-platform’, ‘https://www.googleapis.com/auth/drive’, …
client.get_bucket() returns error: api_request() got an unexpected keyword argument 'extra_api_info' Question: I’m trying to store a newline-delimited JSON string in a GCS bucket using a cloud function, but seeing an error. I start by converting a dataframe to ndjson, then attempt to upload this to my GCS bucket as below. There is more code above …
Reading Data in Vertex AI Pipelines Question: This is my first time using Google’s Vertex AI Pipelines. I checked this codelab as well as this post and this post, on top of some links derived from the official documentation. I decided to put all that knowledge to work, in some toy example: I was planning …
Error uploading file to google cloud storage Question: How should the files on my server be uploaded to google cloud storage? the code I have tried is given below, however, it throws a type error, saying, the expected type is not byte for: the expected type is not byte for: blob.upload_from_file(file.file.read()). Although upload_from_file requires a …
Upload a modified XML file to google cloud storage after editting it with ElementTree (python) Question: I’ve modified a piece of code for merging two or more xml files into one. I got it working locally without using or storing files on google cloud storage. I’d like to use it via cloud functions, which seems …
Return GCS objects created between two dates Question: Is this the most efficient way in python of finding blob storage object created between two dates? def objects_in_date_range(store, start_date, end_date): storage_client = storage.Client() bucket = storage_client.get_bucket(store) bucket_list = storage_client.list_blobs(bucket) for blob in bucket_list: name = blob.name created = blob.time_created # print(type(created), created) # print(type(start_date), start_date) if …
Cloud Storage – Get actual creation date of a file Question: Whenever I upload a file to my Cloud Storage bucket, I expect it to conserve the actual date that file was created. However, whenever I access the file through the storage client API on Python, and check the generation metadata field of the blob, …
How do I split a PDF in google cloud storage using Python Question: I have a single PDF that I would like to create different PDFs for each of its pages. How would I be able to so without downloading anything locally? I know that Document AI has a file splitting module (which would actually …
