Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore conflicting concurrent storage writes of the same data with version="*" #1118

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

ftkg
Copy link
Contributor

@ftkg ftkg commented Oct 25, 2023

Upsert with DO NOTHING will ignore insert on conflicting key, instead of failing. The intended upside is that transactions with conflicting inserts will not have to be retried anymore, saving unnecessary database processing.

Results do not change, as the new query will still return no rows on conflict, hence the batch insert will be aborted with ErrStorageRejectedVersion.

@ftkg ftkg requested a review from redbaron October 25, 2023 14:44
server/core_storage.go Outdated Show resolved Hide resolved
@ftkg ftkg requested a review from redbaron October 25, 2023 21:31
@ftkg ftkg changed the title Ignore conflicting storage inserts with version="*" [WIP] Ignore conflicting storage inserts with version="*" Oct 25, 2023
@ftkg
Copy link
Contributor Author

ftkg commented Oct 26, 2023

New query should return success for concurrent writes of identical objects. Had to use ON CONFLICT UPDATE to return a row for this scenario, that's an extra step but it should be better than retrying (additionally, the previous WHERE NOT EXISTS clause will avoid unnecessary updates for non-concurrent duplicate writes and keep the semantics of rejecting them).

@redbaron
Copy link
Contributor

redbaron commented Oct 26, 2023

UNION ALL LIMIT 1 variant doesn't unnecessary overwrite row just to return it and can be made to work by checking returned row on the Go side: If returned row is the same which is written, then write succeeds.

I see what you mean by "concurrent safe"

@ftkg ftkg changed the title [WIP] Ignore conflicting storage inserts with version="*" Ignore conflicting concurrent storage inserts with version="*" Oct 27, 2023
@ftkg ftkg changed the title Ignore conflicting concurrent storage inserts with version="*" Ignore conflicting concurrent storage writes of the same data with version="*" Oct 27, 2023
@redbaron
Copy link
Contributor

This is a behaviour change which needs to be prominently mentioned in release notes and docs. Previously it was possible to use this operation to create distributed lock/ leader election , now it is still possible but written value must be unique for each node

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants