HOWTO decode a serialized protobuf (in C) within Rust (prost crate)?

[tamama]

Greetings,

We attempt to do the following:

1. serialize proto message (google/protobuf/struct.proto) in C
2. pass the serialized message to Rust via FFI
3. deserialize from Rust (via proto::Message::merge)

However, this throws an exception: "buffer underflow.

What is the best way to achieve this?

Humbly thank everyone in advance!

C:

...
std::string serialized__protobuf__plugin__configuration;
        protobuf__plugin__configuration->SerializeToString(&serialized__protobuf__plugin__configuration);

        auto* plugin = Rust__plugin__input__http_poller__constructor(serialized__protobuf__plugin__configuration.c_str());  // Beware of memory leak here !!!
...

Rust:

#[no_mangle]
pub unsafe fn Rust__plugin__input__http_poller__constructor(serialized__protobuf__plugin__configuration: *const ::libc::c_char) -> *mut SofiaPlugin__input__http_poller {
    let mut ctx: Box<context::Context> = context::Background();

    let serialized__protobuf__plugin__configuration: &str = match ::std::ffi::CStr::from_ptr(serialized__protobuf__plugin__configuration).to_str() {
        Err(_) => return ::std::ptr::null_mut(),
        Ok(serialized__protobuf__plugin__configuration) => serialized__protobuf__plugin__configuration,
    };

    info!("[constructor] serialized__protobuf__plugin__configuration: {}", serialized__protobuf__plugin__configuration);

    let buffer: &[u8] = serialized__protobuf__plugin__configuration.as_bytes();

    let mut protobuf__plugin__configuration: Box<::prost_types::Struct> = Box::new(Default::default());
    match protobuf__plugin__configuration.as_mut()
                                         .merge(serialized__protobuf__plugin__configuration.as_bytes())
    {
        Ok(protobuf__plugin__configuration) => { info!("[OK] protobuf__plugin__configuration - {:#?}", protobuf__plugin__configuration); },
        Err(err) => { info!("[SHIT] err -{:#?}", err)},
    }
    ... ... ...

[tamama]

Greetings again!

Let me answer my own question above.

It turns out that the serialized protobuf contains a lot of '\0', which obviously would mess up your C-string.
(To confirm, you can verify using your-cxx-string.size() with std::strlen(your-c-string))

The following code would then circumvent this situation...

#[no_mangle]
pub unsafe fn Rust__plugin__input__http_poller__constructor(serialized__protobuf__plugin__configuration: *const u8,
                                                            sizeof__serialized__protobuf__plugin__configuration: usize)
        -> *mut SofiaPlugin__input__http_poller
{
    let protobuf__plugin__configuration: Option<::prost_types::Struct>
        = ::prost_types::Struct::decode(::std::slice::from_raw_parts(serialized__protobuf__plugin__configuration,
                                                                     sizeof__serialized__protobuf__plugin__configuration))
                .ok();

    info!("[constructor] protobuf__plugin__configuration: {:?}", protobuf__plugin__configuration);

    let mut ctx: Box<context::Context> = context::Background();
    match SofiaPlugin__input__http_poller::new(ctx.as_ref(), protobuf__plugin__configuration) {
        Err(_) => ::std::ptr::null_mut(),
        Ok(plugin) => Box::into_raw(plugin),
    }
}

You would then see that Prost (Rust) desearializing your C-serialized protocol-buffers correctly!

Cheers!
Tama MA

[caspermeijn]

@tamama I am glad to see you solved your own issue. Please close the issue when you have no further questions.