Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpc: add a query parameter to set initial value for blockingQuery "not found" #12363

Open
dnephin opened this issue Feb 15, 2022 · 1 comment
Labels
type/enhancement Proposed improvement or new feature

Comments

@dnephin
Copy link
Contributor

dnephin commented Feb 15, 2022

As documented in #12110 (comment), the optimization in #12110 requires that the first state store query performed by blockingQuery reports the same "last modified index" as the request queryOpts.MinIndex. Without that kind of stabilization the optimization does not work.

To further improve on this optimization we could add a new query parameter (X-Consul-Index-Not-Found). This query parameter would be a new field on structs.QueryOptions. It would be read in Server.blockingQuery to set the initial value of var notFound bool. This would allow the optimization to work even when a new write has happened between blocking query requests.

For API consumers they would be required to set this value themselves. Internally we could have the agent/cache automatically set this value when appropriate without requiring the user to do so.

@dnephin dnephin added the type/enhancement Proposed improvement or new feature label Feb 15, 2022
@phemmer
Copy link

phemmer commented Jan 16, 2023

I'm currently experiencing an issue which I think is what this issue (#12363) is about, but not completely sure.

My issue is that when performing a blocking KV read on a key (or prefix) which doesn't exist, and the index value in the request isn't the very latest, the response returns immediately rather than blocking, even if no data under the requested path has changed since the specified index.
If the index is frequently updating, this can result in lots of wasted requests until the client manages to get a new request in before the index changes.
This is also an issue with consul-template, as when it is performing a blocking read, after the read times out, it doesn't update the index value before submitting a new blocking query, which given that it sat blocking for 1m, is certainly going to be expired. Though this is more of a consul-template issue, than a consul issue. Just mentioning as the two issues combined exacerbate the situation.

This issue and the referenced comment (#12110 (comment)) use a lot of internal coding references, so I'm not sure this is what's being discussed, and wanted to check before opening an issue on the subject.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement Proposed improvement or new feature
Projects
None yet
Development

No branches or pull requests

2 participants