Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add watch raft log #1354

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

zhangyongding
Copy link

No description provided.

@otoolep
Copy link
Member

otoolep commented Aug 1, 2023

Thanks for your contribution, seems straightforward enough.

So the idea is clients would poll this endpoint? It seems like you also handle the case where the log index doesn't exist, due to a log truncation (which is good). But doesn't this very issue limit the usefulness of this endpoint substantially? Unless the client never misses a log, the client will miss changes. If you want the client to receive a complete record of the changes, it better never go down for any extended period. Do you agree?

@zhangyongding
Copy link
Author

The first time the client gets the latest index number from index 0, the client is not concerned about previous raft log changes. The client then continuously monitors index changes in order to obtain changes in raft logs, and the client continuously updates the index number of the latest raft logs

@otoolep
Copy link
Member

otoolep commented Aug 3, 2023

What is there is a break? For example the client crashes, and a log truncation takes place before the client comes back up. The client will have missed a set of logs in that case. Can you suggest an application where this feature in still useful?

@zhangyongding
Copy link
Author

If the client crashes, when the client restarts, it only needs to obtain the latest data from the database. This scenario is suitable for the client not to persist the data in the database.

@otoolep
Copy link
Member

otoolep commented Aug 3, 2023

OK, thanks for that.

This is something I'll consider after I release 8.0. I need to focus on that work for now.

@zhangyongding
Copy link
Author

You can consider the watch scheme of etcd, which is better

@mauri870
Copy link
Contributor

mauri870 commented Feb 4, 2024

In the advent of a client crash, it would be useful if the client could call this endpoint again upon recovering with a ?last_idx=i query parameter and resume where it had left, this way preventing any changes from being lost. It seems rather simple to implement as well.

Regarding the protocol, http polling seems a bit unoptimized. Server-Sent Events (SSE) might be an alternative, there is no incurred latency, as the connection is persistent. There is also the advantage of not losing events in between the polling intervals.

Overall, your change looks good. Thanks for working on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants