python 3.6 socket pickle data was truncated
Question:
I can not send my numpy array in socket. I use pickle but my client pickle crashes with this error: pickle data was truncated
My server :
I create a numpy array and I want to send in my client with pickle (it’s work)
import socket, pickle
import numpy as np
from PIL import ImageGrab
import cv2
while(True):
HOST = 'localhost'
PORT = 50007
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 4096)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print ('Connected by', addr)
arr = np.array([[0, 1], [2, 3]])
printscreen_pil=ImageGrab.grab(bbox=(10,10,500,500))
img = np.array(printscreen_pil) ## Transform to Array
data_string = pickle.dumps(img)
conn.send(data_string)
msg_recu = conn.recv(4096)
print(msg_recu.decode())
conn.close()
My client
He has my numpy array, but I can not load with pickle. I have this error.
import socket, pickle
import numpy as np
HOST = 'localhost'
PORT = 50007
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
msg_a_envoyer = "hello".encode()
s.send(msg_a_envoyer)
while 1:
data = s.recv(4096)
if not data: break
data_arr = pickle.loads(data)
print (data_arr)
s.close()
Answers:
the problem is that if the size of the pickled data is > 4096 you only get the first part of the pickled data (hence the pickle data was truncated
message you’re getting)
You have to append the data and pickle it only when the reception is complete, for example like this:
data = b""
while True:
packet = s.recv(4096)
if not packet: break
data += packet
data_arr = pickle.loads(data)
print (data_arr)
s.close()
increasing a bytes object is not very performant, would be better to store the parts in a list of objects, then join
, though. Faster variant:
data = []
while True:
packet = s.recv(4096)
if not packet: break
data.append(packet)
data_arr = pickle.loads(b"".join(data))
print (data_arr)
s.close()
In simple words, the file you are trying to load is not complete. Either you have not downloaded it correctly or it’s just that your pickle file is corrupt. You can create a new pickle to solve this issue
I can not send my numpy array in socket. I use pickle but my client pickle crashes with this error: pickle data was truncated
My server :
I create a numpy array and I want to send in my client with pickle (it’s work)
import socket, pickle
import numpy as np
from PIL import ImageGrab
import cv2
while(True):
HOST = 'localhost'
PORT = 50007
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 4096)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print ('Connected by', addr)
arr = np.array([[0, 1], [2, 3]])
printscreen_pil=ImageGrab.grab(bbox=(10,10,500,500))
img = np.array(printscreen_pil) ## Transform to Array
data_string = pickle.dumps(img)
conn.send(data_string)
msg_recu = conn.recv(4096)
print(msg_recu.decode())
conn.close()
My client
He has my numpy array, but I can not load with pickle. I have this error.
import socket, pickle
import numpy as np
HOST = 'localhost'
PORT = 50007
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
msg_a_envoyer = "hello".encode()
s.send(msg_a_envoyer)
while 1:
data = s.recv(4096)
if not data: break
data_arr = pickle.loads(data)
print (data_arr)
s.close()
the problem is that if the size of the pickled data is > 4096 you only get the first part of the pickled data (hence the pickle data was truncated
message you’re getting)
You have to append the data and pickle it only when the reception is complete, for example like this:
data = b""
while True:
packet = s.recv(4096)
if not packet: break
data += packet
data_arr = pickle.loads(data)
print (data_arr)
s.close()
increasing a bytes object is not very performant, would be better to store the parts in a list of objects, then join
, though. Faster variant:
data = []
while True:
packet = s.recv(4096)
if not packet: break
data.append(packet)
data_arr = pickle.loads(b"".join(data))
print (data_arr)
s.close()
In simple words, the file you are trying to load is not complete. Either you have not downloaded it correctly or it’s just that your pickle file is corrupt. You can create a new pickle to solve this issue