How can I use a gRPC "oneof" proto structure in python?

Question

I am building a file storage client/server using python for the client, go for the server, along with gRPC. I have already successfully built the client in go and it works! I am trying to do the same in python now. Today, I have been working on it all day, yet I have made 0 progress -_-. The request I am sending to my server is probably malformed in some way judging by the error message being spit out of the gRPC library:

  File "/Users/xxxxx/Desktop/clients/uploadClient.py", line 60, in upload_file
    for res in stream:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grpc/_channel.py", line 367, in __next__
    return self._next()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grpc/_channel.py", line 361, in _next
    raise self
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
        status = StatusCode.INTERNAL
        details = "Exception serializing request!"
        debug_error_string = "None"
>

Not too helpful. I have not found any helpful documentation and the hours of reading and searching have not payed off yet. The only thing I can rule out so far is that the problem is server related since the go client has been working great.

Here is my proto (note: I did distill this down a bit and renamed some things):

syntax = "proto3";

import "google/protobuf/timestamp.proto";

package upload;

message Chunk {
  message Index {
    uint64 as_uint64 = 1;  
  }

  Index index  = 1;
  bytes sha512 = 2; 
  bytes data   = 3;
}

message Descriptor {
  string author   = 1;  // author
  string label   = 2;  // label
  Format format = 3; //format
}  

enum Format {
    FORMAT_UNKNOWN   = 0; 
    FORMAT_CSV       = 1;
    FORMAT_XML       = 2;
    FORMAT_JSON      = 3;
    FORMAT_PDF       = 4;
}

message UploadFile {
  message ToClient {
    oneof details {
      Finished finished = 1;
    }
  }

  message ToService {
    oneof details {
      Descriptor descriptor = 1;
      Chunk chunk           = 2;
    }
  }
}

.
service FileService {
  rpc Upload(stream UploadFile.ToService) returns (stream UploadFile.ToClient);
}

Here is the code (note: I did distill this down a bit and renamed some things):

import s_pb2 as s
import s_pb2_grpc as s_grpc

token = 'xxxx'
url = 'xxxx:433'
requestMetadata = [('authorization', 'Bearer ' + token)]

def create_stub():
    creds = grpc.ssl_channel_credentials()
    channel = grpc.secure_channel(url, creds)
    return s_grpc.UploadManagerStub(channel)

def upload_file(src, label, fileFormat):
    stub = create_stub()

    stream = stub.Upload(
        request_iterator=__upload_file_iterator(src, label, fileFormat),
        metadata=requestMetadata
    )

    for res in stream:
        print(res)

    return stream

def __upload_file_iterator(src, name, fileFormat, chunk_size = 1024):

    def descriptor():
        to_service = s.UploadFile.ToService

        to_service.descriptor = s.Descriptor(
            label=label,
            format=fileFormat
        )

        return to_service

    yield descriptor()

Yes, I know my iterator is only returning 1 thing, I removed some code to try to isolate the problem.

gRPC is not my strongest skills and I would like to believe that my pea brain is just missing something obvious.

All help is appreciated!

Asked By: 88jayto

||

Source

Answer 1

Wrapping the “oneof” details in the constructor fixed the problem:

def chunk(data, index):
    return s.UploadFile.ToService(
        chunk=s.Chunk(
            index=s.Chunk.Index(as_uint64=index),
            data=data,
            sha512=hashlib.sha512(data).digest()
        )
    )

def descriptor(name, fileFormat):
    return s.UploadFile.ToService(
        descriptor=s.Descriptor(
            name=name,
            format=fileFormat
        )
    )

Answered By: 88jayto

How can I use a gRPC "oneof" proto structure in python?

Question:

Answers: