Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new libp2p subprotocol for msg-ack flow #6270

Open
14 tasks
tolbrino opened this issue May 15, 2024 · 0 comments
Open
14 tasks

Add new libp2p subprotocol for msg-ack flow #6270

tolbrino opened this issue May 15, 2024 · 0 comments
Assignees
Labels
epic An issue which tracks multiple issues status:product-backlog

Comments

@tolbrino
Copy link
Contributor

tolbrino commented May 15, 2024

Currently the entire msg + ack workflows are split into separate unidirectional subprotocols issues and handled as separate code paths. This causes several complications:

  1. it is difficult to pair Ack for each Msg
  2. in case of missing Ack the Msg protocol is still considered successful, but it should fail
  3. there is no automatic pairing of Ack and Msg even for failed messages
  4. workflows requiring bi-directional logic can not be easily used over the msg + ack combination and the implementation is overly complex.
  5. introduce sampled message latency tracking over the new bi-directional protocol

It'd be beneficial to finally unite both subprotocols into a single one and implement relevant fixes to known subprotocol issues related to HOPR protocol.

Description

The new msg + ack protocol will have a single RequestId allowing it to be trackable together. That in turn allows to fix some of the major issues of the current design, including:

  • Use the new msg/ack protocol to randomly sample the peer connection instead of the heartbeat protocol, when messages pass through it
  • Always ack the msg request, even on msg failure, to increase the privacy
  • Simplify the packet processing pipeline to abstract away from the non-transport related behavior (e.g. tickets, chain ops...)
  • Update the code to use lazy initialization where possible and optimize background processes with true parallelization (e.g. rayon).
  • Optimize the large data object pass-through in the pipelines to improve throughput
  • Support the legacy protocol and allow the new protocol to be used/attempted to use by default.

Relevant issues

Definition of DONE

  • A new msg + ack protocol is defined in term of the RequestResponse libp2p behavior
  • The new protocol is the default protocol
  • The legacy protocol is supported in a backwards compatible manner
  • Randomly selected latency measurements are taken from the new protocol
  • The process pipeline is streamlined and simplified
  • The new abstractions are in place to allow removing non-transport related code from the transport layer and hide those behind a trait
  • Lazy initialization is used wherever possible
  • Packet processing pipeline uses a parallelization engine
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
epic An issue which tracks multiple issues status:product-backlog
Projects
None yet
Development

No branches or pull requests

2 participants