Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document how users can preserve transactionality in their networking layer #303

Open
tantaman opened this issue Aug 3, 2023 · 0 comments

Comments

@tantaman
Copy link
Collaborator

tantaman commented Aug 3, 2023

The core of cr-sqlite drops metadata aggressively to keep DB size down, even when offline for prolonged periods of time or doing thousands of writes per second.

This choice, however, impacts how transactions are synced (documented here: https://vlcn.io/blog/how-crsqlite-transactions-work-today) but users can maintain transactional sync in the network layer if desired.

The basic idea is that the network layer would package changes from last_seen db_version to current db_version in a single block and peers would write that block in a single transaction.

Blocks can be as small as a single db_version or as large as the user wants, so long as:

  1. The entire block is applied at once
  2. Blocks are applied in-order

If blocks have not been sent to a peer yet, they can be merged together to save space. There is a balance here between:

  1. having really large blocks but less overall data
  2. having really small blocks but more overall data
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant