Push Messages from AWS Lambda to Kafka

Question

I have a kafka machine running in AWS which consists of several topics.
I have the following Lambda function which Produces a message and push that to one of the kafka topic.

import json from kafka
import KafkaClient from kafka
import SimpleProducer from kafka
import KafkaProducer

def lambda_handler(event, context):
    kafka = KafkaClient("XXXX.XXX.XX.XX:XXXX")
    print(kafka)
    producer = SimpleProducer(kafka, async = True)
    print(producer)
    task_op = {
        "'message": "Hai, Calling from AWS Lambda"
    }
    print(json.dumps(task_op))
    producer.send_messages("topic_atx_ticket_update",json.dumps(task_op).encode('utf-8'))
    print(producer.send_messages)
    return ("Messages Sent to Kafka Topic")

But I see messages are not pushed as i expected.

Note: No Issues in Roles and Policies, Connectivity.

Asked By: RajNikhil Marpu

||

Source

Answer 1

While Creating a Kafka Producer object,

producer = SimpleProducer(kafka, async=True)

“async” String should be False, like

producer = SimpleProducer(kafka, async=False)

Then,

you can send the Kafka Message to a topic from AWS Lambda.

Answered By: RajNikhil Marpu

Push Messages from AWS Lambda to Kafka

Question:

Answers: