Endianness of UUID and protobuf

[viniciusccarvalho]

Hi there, I just wanted to call out for an issue I'm experiencing, may or may not be a bug.

Trying to parse the returned value from UUID of the protocol in Java, always return a mismatch UUID. So I took a peak into the code and found this line:

return &control.UUID{Low: proto.Uint64(binary.LittleEndian.Uint64(id[:8])), High: proto.Uint64(binary.LittleEndian.Uint64(id[8:]))}

I'm not that familiar with protobuf, but from discussions around the web, it seems that it should take care of endianness of the bytes, and each client convert to its native implementation.

And because this is forcing little endian value, when java clients try to read them, I'm forced to read as BigIntegers, convert to bytes and revert them in order to construct the proper UUID string back.

Is this really the expected behavior? Or should it be a more interoperable way to parse this info?

Regards

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: https://www.pivotaltracker.com/story/show/93019308.

[jmtuley]

@viniciusccarvalho,

I want to answer your question in two parts:

1. Yes, it's the expected behavior that we use a little-endian encoding for the UUID. The reason that it is done explicitly is that the internal representation is a 16-element byte array, which then needs to be packed into a pair of 64-bit integer. When doing a conversion like that, endianness must be considered.
2. No, it wasn't our goal to force Java developers through ugly hoops.

This also isn't the first time we've run into concerns with interoperability of the UUID field. While representing the 128 bits of the UUID as a pair of 64-bit integers is certainly the most compact form possible, it's not very usable. Endianness, compatibility with client libraries, and clarity all seem to be issues around this form.

My crystal ball is broken, so this is a guess at the future and not a promise, but I think in a future version of the protocol, we'll transmit the UUID in the usual hexadecimal string form (i.e. xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx). It takes a few more bytes, but its intent is clear, it's human-readable and every library we've found knows how to parse it.

Thanks for pointing this out; it gives us an even stronger reason to consider the string form of the UUID.

[viniciusccarvalho]

@jmtuley

Thank you for the clarification. I understand the need to compact the message, but as you described it can be a bit painful for developers. I would really welcome a string uuid in exchange for the extra bytes of overhead.

Could it be that for now the docs just state on the UUID type that is using little endian order? I think that alone could save others a few bumps when integrating.

Thanks

[jmtuley]

Absolutely. Better documentation is a good idea.

[jmtuley]

I'm going to close this issue, since the documentation will live over at cloudfoundry/dropsonde-protocol. The existing tracker story will continue to reflect the progress of the work.

[jmtuley]

Updated documentation on the Dropsonde Protocol repository. See for events and control messages.

[CAFxX]

protocolbuffers/protobuf#2224 upstream issue (native UUID type)