ZMQ Python PUB/SUB works but not my C++ Subscriber with Python Publisher

Question:

I am using Ubuntu 18.04, Qt 5.12, and libzmq.so.5.1.5. I have 2 very simple Python 3.6 scripts, using pyzmq 18.0.1, one implements ZMQ PUB and one implements ZMQ SUB. These Python scripts work find and data is transmitted from one to the other on the localhost loopback network. I am trying to implement the subscriber in Qt 5.12, and the subscriber always blocks on receive when I send the same data using the same Python publisher script. I’ve pasted the complete code below – how I can I get ZMQ with Qt 5.12 to actually received data? Thanks.

The output of my C++ app looks successful:

"Server IP Address determined to be: 127.0.0.1"
"Connecting to: tcp://127.0.0.1:5555"
"Attempted Connect: 0"
"SetSockOpt: 0"
"GetSockOpt: 0"
Done setting up socket
Waiting

Python 3 PUB:

#!/usr/bin/python3

import time
import zmq
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://127.0.0.1:5555")

# Allow clients to connect before sending data
time.sleep(3)
while True:
    socket.send_pyobj({1:[1,2,3]})
    time.sleep(1)

Python 3 SUB:

#!/usr/bin/python3

import zmq
context = zmq.Context()
socket = context.socket(zmq.SUB)
# We can connect to several endpoints if we desire, and receive from all.
socket.connect("tcp://127.0.0.1:5555")

# We must declare the socket as of type SUBSCRIBER, and pass a prefix filter.
# Here, the filter is the empty string, which means we receive all messages.
# We may subscribe to several filters, thus receiving from all.
socket.setsockopt(zmq.SUBSCRIBE, b'')

while True:
    message = socket.recv_pyobj()
    print(message.get(1)[2])

Qt 5.12 Subscriber Code:

#include <QCoreApplication>
   
// Std includes
#include <stdio.h>
#include <unistd.h>
#include <assert.h>

// Qt
#include <QDebug>
#include <QFile>

// ZeroMQ Includes
#include <zmq.h>

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
   
    //QString m_ServerIP = QString("*");
    QString m_ServerIP = QString("127.0.0.1");
    QString m_ServerPort = QString("5555");

    qDebug() << QString("Server IP Address determined to be: %1").arg(m_ServerIP);

    void* m_Context = zmq_ctx_new();
    assert (m_Context);
    void* m_Subscriber = zmq_socket (m_Context, ZMQ_SUB);
    assert (m_Subscriber);
    int rc = -1;
    unsigned int fd = 0;
    do {

        const char *filter = std::string("").c_str();
        QString ipAndPort = QString("tcp://%1:%2").arg(m_ServerIP).arg(m_ServerPort);

        qDebug() << QString("Connecting to: %1").arg(ipAndPort);

        rc = zmq_connect(m_Subscriber, ipAndPort.toStdString().c_str());

        qDebug() << QString("Attempted Connect: %1").arg(rc);

        rc = zmq_setsockopt(m_Subscriber, ZMQ_SUBSCRIBE,filter, strlen (filter));

        qDebug() << QString("SetSockOpt: %1").arg(rc);

        size_t fd_size = sizeof(fd);

        rc = zmq_getsockopt(m_Subscriber,ZMQ_FD,&fd,&fd_size);

        qDebug() << QString("GetSockOpt: %1").arg(rc);

    }
    while ( rc < 0 );

    qDebug() << "Done setting up socket";

    while ( true) {

        zmq_msg_t message;
        zmq_msg_init(&message);
        qDebug() << "Waiting";
        zmq_recvmsg(m_Subscriber, &message, 0);
        size_t size = zmq_msg_size (&message);

        qDebug() << QString("Message Size: %1").arg(size);

        char *string = static_cast<char*>(malloc(size + 1));
        memcpy (string, zmq_msg_data(&message), size);
        zmq_msg_close (&message);
        string [size] = 0;
        if (string) {
            QByteArray frame = QByteArray::fromBase64(QByteArray(string));
            free(string);

            qDebug() << QString("Debug RX Frame Size: %1").arg(frame.size());

            QFile output("/tmp/abcd.jpeg");
            if ( output.open(QIODevice::WriteOnly) ) {
                output.write(frame);
                output.close();
            }

        }

    }

    return a.exec();
}
Asked By: PhilBot

||

Answers:

You have the following errors:

  • You do not need QCoreApplication since you are not using signals, slots, events, etc. that need an event-loop.

  • You are using the C api of zmq, there is a compatible api for C++, and to use it use the header <zmq.hpp>.

  • When you use send_pyobj() python it uses pickle to transmit the data, but in C++ there is no method to revert the data. Instead you should send the data with send_json() from python.

Considering the above, the solution is:

*.py

#!/usr/bin/python3

import time
import zmq
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://127.0.0.1:5555")

# Allow clients to connect before sending data
time.sleep(3)
while True:
    socket.send_json({1:[1,2,3]})
    time.sleep(1)

main.cpp

#include <QString>
#include <QDebug>
#include <QUrl>
#include <QThread>

#include <zmq.hpp>

int main(void) {

    QString m_ServerIP = QString("127.0.0.1");
    int m_ServerPort = 5555;

    qDebug() << QString("Server IP Address determined to be: %1").arg(m_ServerIP);

    QUrl url;
    url.setScheme("tcp");
    url.setHost(m_ServerIP);
    url.setPort(m_ServerPort);

    zmq::context_t context(1);
    zmq::socket_t subscriber (context, ZMQ_SUB);
    subscriber.connect(url.toString().toStdString());
    subscriber.setsockopt( ZMQ_SUBSCRIBE, "", 0);

    while(true) {
        zmq_msg_t in_msg;
        zmq_msg_init(&in_msg);
        zmq::message_t  message;
        subscriber.recv(&message);
        qDebug() << QString("Message Size: %1").arg(message.size());
        QByteArray ba(static_cast<char*>(message.data()), message.size());
        qDebug()<< "message" << ba;
        QThread::sleep(1);
    }
}

Output:

"Server IP Address determined to be: 127.0.0.1"
"Message Size: 13"
message "{"1":[1,2,3]}"
"Message Size: 13"
message "{"1":[1,2,3]}"
"Message Size: 13"
message "{"1":[1,2,3]}"
"Message Size: 13"
message "{"1":[1,2,3]}"
"Message Size: 13"
message "{"1":[1,2,3]}"

You can decode the data using QJsonDocument. I also recommend you to review the following examples in C++ of zmq.


If you use socket.send_pyobj({1:[1,2,3]}) you get the following that is the result of pickle:

"Server IP Address determined to be: 127.0.0.1"
"Message Size: 20"
message "x80x03}qx00Kx01]qx01(Kx01Kx02Kx03""es."
"Message Size: 20"
message "x80x03}qx00Kx01]qx01(Kx01Kx02Kx03""es."
"Message Size: 20"
message "x80x03}qx00Kx01]qx01(Kx01Kx02Kx03""es."
"Message Size: 20"
message "x80x03}qx00Kx01]qx01(Kx01Kx02Kx03""es."
"Message Size: 20"
message "x80x03}qx00Kx01]qx01(Kx01Kx02Kx03""es."
Answered By: eyllanesc
Categories: questions Tags: , , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.