Towards Zero-Copy Binary (De)Serialization: TypeScript ↔ Rust

---

Slides for this talk are also available here.

Abstract

JSON is the de facto standard for sharing data between services, but its neither type-safe nor cheap to parse. Enter the world of zero-copy binary protocols for Rust and TypeScript applications. Starting from commonly found - but non trivial - datatypes and their JSON schema definition, we will discuss how to model them in Cap'n Proto: a proven, zero-copy binary protocol used at Cloudflare, and crafted by the creator of Protobuf. Ready to level up your cross-language data handling prowess?

Requirements

• Rust 1.73.0 or superior*
• Deno 1.36.0 or superior*

(*) These are the versions used to develop this repository. Older versions might work as well, but they haven't been tested.

Furthermore, you'll need to install:

• wasm-bindgen, via:

  cargo install -f wasm-bindgen-cli@0.2.87

• capnp, via:

  brew install capnp

  or

  apt-get install capnproto

• capnpc-ts, the TypeScript bindings for Cap'n Proto, via:

  deno install --allow-env --allow-read --allow-write -n capnpc-ts "https://deno.land/x/capnpc/mod.ts"

You'll also need to enable to wasm32-unknown-unknown target for Rust, via:

rustup target add wasm32-unknown-unknown

How to Run

• Compile the Rust codebase and generate bindings for Cap'n Proto and WebAssembly via:

  ./build.sh

• Run the ./src/event.ts demo via:

  deno task event

Additional

You can use capnp to convert a Cap'n Proto binary to JSON. For instance, given the ./src/capnp/event.capnp Cap'n Proto schema containing the Event struct definition, and the example binary message at ./bin/event.bin, you can run

capnp convert binary:json ./src/capnp/event.capnp Event < ./bin/event.bin | jq

which should yield

{
  "name": "trivago",
  "year": 2023
}

👤 Author

Alberto Schiabel

• Twitter: @jkomyno
• Github: @jkomyno

Please consider supporting my work by following me on Twitter and starring my projects on GitHub. I mostly post about TypeScript, Rust, and WebAssembly. Thanks!

📝 License

Built with ❤️ by Alberto Schiabel. This project is MIT licensed.