SSL pickle error when running multiprocess inside class's method and websocket

Question

I am creating a phyton object that establishes a websocket connection.
Depending the message that I receive, I would like to run a class’s method on a child-process using multiprocessing. However, I get the error cannot pickle 'SSLSocket' object. Why does it happen?

The following code reproduces the error.

import websocket
import json
from multiprocessing import Process


def on_close(ws, close_status_code, close_msg):
    print('closed')


def on_error(ws, error):
   print(error)


class ObClient():
    def __init__(self):
        self.ws = None

    def on_open(self, ws):
        subscribe_message = {"method": "SUBSCRIBE",
                             "params": 'btcusdt@depth5@1000ms',
                             "id": 1}
        ws.send(json.dumps(subscribe_message))

    def func(self):
        print('hello')


    def on_message(self, ws, message):
        process = Process(target=self.func)
        process.start()
        process.join()

    def connect(self):
        self.ws = websocket.WebSocketApp('wss://stream.binance.com:443/stream', on_close=on_close, on_error=on_error,
                                         on_open=self.on_open, on_message=self.on_message)
        self.ws.run_forever()

if __name__ == '__main__':
    client = ObClient()
    client.connect()

Asked By: apt45

||

Source

Answer 1

To create a Process, Python pickles any objects used by the Process. In your case, the target function is self.func, a member function of ObClient. Member functions have access to all of the object’s variables, one of which is self.ws, an instance of WebSocket. Therefore Python tries to pickle WebSocket, which uses operating system resources and therefore cannot be pickled. That’s why you get an error.

One way to fix this is to move func to a different class or to make it a standalone function. Then it will not drag in all the member variables of ObClient.

Answered By: Paul Cornelius

SSL pickle error when running multiprocess inside class's method and websocket

Question:

Answers: