Simple IPC between C++ and Python (cross platform)
Question:
I have a C++ process running in the background that will be generating ‘events’ infrequently that a Python process running on the same box will need to pick up.
- The code on the C side needs to be as lightweight as possible.
- The Python side is read-only.
- The implementation must be cross-platform.
- The data being sent is very simple.
What are my options?
Thanks
Answers:
How complex is your data? If it is simple I would serialize it as a string. If it was moderately complex I would use JSON. TCP is a good cross-platform IPC transport. Since you say that this IPC is rare the performance isn’t very important, and TCP+JSON will be fine.
Use zeromq, it’s about as simple as you can get.
Another option is to just call your C code from your Python code using the ctypes module rather than running the two programs separately.
zeromq — and nothing else. encode the messages as strings.
However, If you want to get serialiazation from a library use protobuf it will generate classes for Python and C++. You use the SerializeToString() and ParseFromString() functions on either end, and then pipe the strings via ZeroMq.
Problem solved, as I doubt any other solution is faster, and neither will any other solution be as easy to wire-up and simple to understand.
If want to use specific system primitives for rpc such as named pipes on Windows and Unix Domain Sockets on unix then you should look at Boost::ASIO. However, unless you have (a) a networking background, and (b) a very good understanding of C++, this will be very time consuming
I will say you create a DLL that will manage the communication between the two. The python will load DLL and call method like getData() and the DLL will in turn communicate with process and get the data.
That should not be hard.
Also you can use XML file or SQLite database or any database to query data. The daemon will update DB and Python will keep querying. There might be a filed for indicating if the data in DB is already updated by daemon and then Python will query.
Of course it depends on performance and accuracy factors!
You can use Google GRPC for this
I have a C++ process running in the background that will be generating ‘events’ infrequently that a Python process running on the same box will need to pick up.
- The code on the C side needs to be as lightweight as possible.
- The Python side is read-only.
- The implementation must be cross-platform.
- The data being sent is very simple.
What are my options?
Thanks
How complex is your data? If it is simple I would serialize it as a string. If it was moderately complex I would use JSON. TCP is a good cross-platform IPC transport. Since you say that this IPC is rare the performance isn’t very important, and TCP+JSON will be fine.
Use zeromq, it’s about as simple as you can get.
Another option is to just call your C code from your Python code using the ctypes module rather than running the two programs separately.
zeromq — and nothing else. encode the messages as strings.
However, If you want to get serialiazation from a library use protobuf it will generate classes for Python and C++. You use the SerializeToString() and ParseFromString() functions on either end, and then pipe the strings via ZeroMq.
Problem solved, as I doubt any other solution is faster, and neither will any other solution be as easy to wire-up and simple to understand.
If want to use specific system primitives for rpc such as named pipes on Windows and Unix Domain Sockets on unix then you should look at Boost::ASIO. However, unless you have (a) a networking background, and (b) a very good understanding of C++, this will be very time consuming
I will say you create a DLL that will manage the communication between the two. The python will load DLL and call method like getData() and the DLL will in turn communicate with process and get the data.
That should not be hard.
Also you can use XML file or SQLite database or any database to query data. The daemon will update DB and Python will keep querying. There might be a filed for indicating if the data in DB is already updated by daemon and then Python will query.
Of course it depends on performance and accuracy factors!
You can use Google GRPC for this