google-cloud-python

Efficiently write a Pandas dataframe to Google BigQuery

Efficiently write a Pandas dataframe to Google BigQuery Question: I’m trying to upload a pandas.DataFrame to Google Big Query using the pandas.DataFrame.to_gbq() function documented here. The problem is that to_gbq() takes 2.3 minutes while uploading directly to Google Cloud Storage takes less than a minute. I’m planning to upload a bunch of dataframes (~32) each …

Total answers: 3

Upload python object to Google Cloud Storage without saving it to file

Upload python object to Google Cloud Storage without saving it to file Question: I am clipping images in Python 2.7 on Windows 10 and would like to send them to Google’s Cloud Vision API. For images saved locally, I can send them using the google.cloud python module like this: blob.upload_from_filename(filename=path) Documented here I would rather …

Total answers: 2

Difficulty comparing generated and google cloud storage provided CRC32c checksums

Difficulty comparing generated and google cloud storage provided CRC32c checksums Question: I am attemptting to get a CRC32c checksum on my local file so I can compare it to the blob.crc32c provided by the gcloud library. Google says I should be using the crcmod module in order to actually calculate CRC32c hashes of my data. …

Total answers: 3

Set metadata in Google Cloud Storage using gcloud-python

Set metadata in Google Cloud Storage using gcloud-python Question: I am trying to upload a file to Google Cloud Storage using gcloud-python and set some custom metadata properties. To try this I have created a simple script. import os from gcloud import storage client = storage.Client(‘super secret app id’) bucket = client.get_bucket(‘super secret bucket name’) …

Total answers: 3

How to use gcloud commands programmatically via Python

How to use gcloud commands programmatically via Python Question: The Google documentation is a little generic on this topic and I find it hard to get around the different APIs and terms they’re using, so I’m wondering if someone could point me to the right direction. I’m looking for a way to call the gcloud …

Total answers: 4