protobuf MessageToJson removes fields with value 0

Question

I’m writing a Python script that receives protobufs, converts them to json objects, and pushes them to another service. I use json.loads(MessageToJson(protobuf)) to convert the protobuf to a python dictionary object. Later I convert it back to json with json.dumps(dictionary).

I have a proto with an optional enumerated field such as:

enum C_TYPE
{
    AB = 0;
    BC = 1;
    CD = 2;
}

When I receive a proto with a field designated as BC everything works as I expect it. When I receive a proto with a field designated AB that field gets ignored — it does not turn up in the python dictionary or subsequent json dump. A workaround I have found is to use json.loads(MessageToJson(protobuf, including_default_value_fields=True)) but that will create default values for all missing fields, not just the ones that have a 0 enumeration. It implies that the field with enumeration 0 is missing – but it’s not!

What is the correct way to retrieve the value of the enumeration field when it is set to 0?

Asked By: oregano

||

Source

Answer 1

There is no correct way, I’m defining my protobufs incorrectly. For enumerated fields, the first value is the default value. This means if a protobuf comes through without a set value, it is set to the default value and, when converted to json, ignored (unless you want to keep all default values.)

Thus, it is recommended use throw-away names for the default value to be able to properly distinguish when it has been set. i.e. I should define my protobuf as:

enum C_TYPE
{
    NONE = 0;
    AB = 1;
    BC = 2;
    CD = 3;
}

From Protobuf Documentation on Optional Fields And Default Values:

For enums, the default value is the first value listed in the enum’s
type definition. This means care must be taken when adding a value to
the beginning of an enum value list.

Additionally from an issue on golang/protobuf:

This is working as intended. proto3 zero-values are omitted in the
JSON format too. The zero-value should be a “throwaway” value: it’s
also what you will see if the sender of a serialized message sets the
field to an invalid or unrecognized value.

Answered By: oregano

Answer 2

In protobuf in C++ it is possible to add an option to always print primitive fields.
Adding this option in the MessageToJson conversion makes protobuf add the parameters even though they are set to zero.

google::protobuf::util::JsonPrintOptions options;
options.always_print_primitive_fields = true;

Answered By: user1728363

Answer 3

In Python, you may set including_default_value_fields argument to True to get the value of all fields.

Answered By: user1507435

protobuf MessageToJson removes fields with value 0

Question:

Answers: