Google cloud PubSub service not working (Python)

Question

I am trying to use the pub sub service on my python application. When I am running the code it get stuck on the last publisher line for some reason and the code never end. The subscriber seems fine.
Does someone know what is wrong with my code?

Publisher:

import os
from google.cloud import pubsub_v1

credentials_path = 'PATH/TO/THE/KEY.JSON'
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = credentials_path


publisher = pubsub_v1.PublisherClient()
topic_path = 'projects/PROJECT_NAME/topics/TOPIC_NAME'

# simple garbage text to check if it's working
data = 'A garden sensor is ready!'
data = data.encode('utf-8')
attributes = {
    'sensorName': 'garden-001',
    'temperature': '75.0',
    'humidity': '60'
}

future = publisher.publish(topic_path, data, **attributes)
print(f'published message id {future.result()}') # here it is just waiting forever

Subscriber:

import os
from google.cloud import pubsub_v1
from concurrent.futures import TimeoutError

credentials_path = 'PATH/TO/THE/KEY.JSON'
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = credentials_path


subscriber = pubsub_v1.SubscriberClient()
subscription_path = 'projects/PROJECT_NAME/subscriptions/SUBSCRIPTION_NAME'


def callback(message):
    print(f'Received message: {message}')
    print(f'data: {message.data}')

    if message.attributes:
        print("Attributes:")
        for key in message.attributes:
            value = message.attributes.get(key)
            print(f"{key}: {value}")

    message.ack()


streaming_pull_future = subscriber.subscribe(
    subscription_path, callback=callback)
print(f'Listening for messages on {subscription_path}')


# wrap subscriber in a 'with' block to automatically call close() when done
with subscriber:
    try:
        streaming_pull_future.result()
    except TimeoutError:
        streaming_pull_future.cancel()
        # block until the shutdown is complete
        streaming_pull_future.result()

Asked By: bob

||

Source

Answer 1

Google provides decent documentation for using its services including Pub/Sub including a basic Python example that would have helped you avoid your problem.

Aside: your publisher and subscriber snippets set GOOGLE_APPLICATION_CREDENTIALS statically within the code. Don’t do this! Set the environment variable before running the code. This way, you can revise the value without changing the code but, more importantly, the value can be set by the runtime e.g. Compute Engine.

Here’s a working example based on your code using Application Default Credentials obtained from the environment:

Q="74535931"

BILLING="[YOUR-BILLING-ID]"
PROJECT="$(whoami)-$(date %y%m%d)-${Q}"

gcloud projects create ${PROJECT}
gcloud beta billing projects link ${PROJECT} 
--billing-account=${BILLING}

gcloud services enable pubsub.googleapis.com 
--project=${PROJECT}

ACCOUNT=tester
EMAIL=${ACCOUNT}@${PROJECT}.iam.gserviceaccount.com

gcloud iam service-accounts create ${ACCOUNT} 
--project=${PROJECT}

gcloud iam service-accounts keys create ${PWD}/${ACCOUNT}.json 
--iam-account=${EMAIL}

gcloud projects add-iam-policy-binding ${PROJECT} 
--member=serviceAccount:${EMAIL} 
--role=roles/pubsub.editor

export GOOGLE_APPLICATION_CREDENTIALS=${PWD}/${ACCOUNT}.json
export PROJECT
export PUB="pub"
export SUB="sub"

gcloud pubsub topics create ${PUB} 
--project=${PROJECT}

gcloud pubsub subscriptions create ${SUB} 
--topic=${PUB} 
--project=${PROJECT}

publish.py:

import os
from google.cloud import pubsub_v1

project = os.getenv("PROJECT")
topic = os.getenv("PUB")

topic_path = f"projects/{project}/topics/{topic}"

data = 'A garden sensor is ready!'
data = data.encode('utf-8')
attributes = {
    'sensorName': 'garden-001',
    'temperature': '75.0',
    'humidity': '60'
}

publisher = pubsub_v1.PublisherClient()
future = publisher.publish(topic_path, data, **attributes)
print(f'published message id {future.result()}')

subscribe.py:

import os
from google.cloud import pubsub_v1
from concurrent.futures import TimeoutError


project=os.getenv("PROJECT")
subscription=os.getenv("SUB")
subscription_path = f"projects/{project}/subscriptions/{subscription}"


def callback(message):
    print(f'Received message: {message}')
    print(f'data: {message.data}')

    if message.attributes:
        print("Attributes:")
        for key in message.attributes:
            value = message.attributes.get(key)
            print(f"{key}: {value}")

    message.ack()


subscriber = pubsub_v1.SubscriberClient()

streaming_pull_future = subscriber.subscribe(
    subscription_path, callback=callback)
print(f'Listening for messages on {subscription_path}')

with subscriber:
    try:
        streaming_pull_future.result()
    except TimeoutError:
        streaming_pull_future.cancel()
        # block until the shutdown is complete
        streaming_pull_future.result()

Run python3 subscribe.py:

python3 subscribe.py
Listening for messages on projects/{project}/subscriptions/{sub}
Received message: Message {
  data: b'A garden sensor is ready!'
  ordering_key: ''
  attributes: {
    "humidity": "60",
    "sensorName": "garden-001",
    "temperature": "75.0"
  }
}
data: b'A garden sensor is ready!'
Attributes:
humidity: 60
temperature: 75.0
sensorName: garden-001

And in a separate window python3 publish.py:

python3 publish.py
published message id 1234567890123456

Answered By: DazWilkin

Google cloud PubSub service not working (Python)

Question:

Answers: