Grpc not increasing max size of the message: received message larger than max

Question:

I am using grpc to send some pretty large messages (the parameters of a machine learning model over the network). The problem is that I am getting the following error when I make a grpc call:

grpc: received message larger than max (261268499 vs. 4194304)

As suggested in other posts I tried to increase the max message size on the channel and the grpc server, but I keep getting the same error. Any idea on how to get this to work?

My code for the server:

maxMsgLength = 1024 * 1024 * 1024
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10),
                     options=[('grpc.max_message_length', maxMsgLength),
                              ('grpc.max_send_message_length', maxMsgLength),
                              ('grpc.max_receive_message_length', maxMsgLength)])

The client:

maxMsgLength = 1024 * 1024 * 1024
channel = grpc.insecure_channel(ip_port, 
                                options=[('grpc.max_message_length', maxMsgLength),
                                         ('grpc.max_send_message_length', maxMsgLength),
                                         ('grpc.max_receive_message_length', maxMsgLength)])

Edit:

Not a solution, but maybe gives a little bit more insight into the problem. For some reason, if I set the max message size to 1024 * 1024 * 1024 it ends up defaulting to 4194304 as the error message implies. Not really sure why that happens. But anyways, I tried reducing the max message size to 1024 * 1024 * 200 and it shows the correct max message size in the error message (209715200). It seems like there is a problem where grpc is not setting the max message size properly. Not sure how to get around this though.

The maximum number I can use where the error message shows the proper max value is 2^28. If I put a max message size of 2^29 it defaults to 4194304.

Asked By: MUAS

||

Answers:

I encountered this problem as well.

A solution is to set the option to -1, as described here:

https://github.com/grpc/grpc/blob/9a7e77318bd881f00033e20115bc4254ebf5eb0a/include/grpc/impl/grpc_types.h#L150-L152

Answered By: Henry
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.