[IMPROVEMENT] General stability and bug fixes. #3999
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
derekcollison
force-pushed
the
interest-stresser
branch
3 times, most recently
from
003b64a
to
3581673
Compare
1. Fixed a bug that would process a removal of a message after the message block was closed. 2. Improved removal of non-existant message when we know the store is empty. 3. Improved last write index size tracking when opening the file descriptor after being closed. 4. Improved Compact() by not loading messages for last block twice. 5. Improved Compact() determination of calling purge by determing last sequence under write lock. 6. Improved Compact() by only compacting underlying message block if over certain size threshold. 7. Improved Compact() by writing the index file if needed while still holding lock avoiding an unecessary re-lock. 8. Improved Compact() by not calling out to upper layers on no messages being purged. 9. Fixed a bug in Compact() that would not delete members from a block's delete map. 10. Fixed a bug in reset() when a callback was not registered (raft logs) which avoiding msg block cleanup. 11. Improved consumer store Update() call for when to avoid an outdated update. Signed-off-by: Derek Collison <derek@nats.io>
1. If reset ignore Applied() that are greater then our commit. 2. Improved StepDown() by placing at back of queue if preferred. 3. Improved handling of leadership transfer during StepDown(). 4. Do not store EntryLeaderTransfer records on disk. 5. Remove un-needed processing of older terms. 6. If append entry has higher term, also inherit pterm. 7. Only inherit a candidate's term if we decide to vote for them. Signed-off-by: Derek Collison <derek@nats.io>
1. Do not process an ack if we are closed. 2. When checking for needing an ack for a given consumer, hold lock entire time. 3. During recovery and restarts we check if we need to replay acks to the parent stream. Signed-off-by: Derek Collison <derek@nats.io>
1. During ackMsg processing hold write lock to block concurrent access. 2. Check for presence of preAcks before and force removal if present. 3. Rework check for orphan msgs on startup to use checkStateForInterestStream(). Signed-off-by: Derek Collison <derek@nats.io>
…rive before the actual msgs. 1. If we are retention based, make sure our consumers are running before entering into monitorStream logic. 2. If we skip messages and are interest based, make sure we check for a preAck state. 3. On finalization of recovery for consumers have them check against the interest based stream. 4. Do not process ack state updates if consumer is closed and shutting down. 5. When processing final state for a stream after upper layer catchup, check all attached consumers for ack skew. 6. During catchup of stream messages consult preAck state and skip messages as needed. Signed-off-by: Derek Collison <derek@nats.io>
…rs during rolling restarts. Signed-off-by: Derek Collison <derek@nats.io>
… were getting and advantage on that after server restart. This change speeds up raft layer more to avoid timeouts. Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Derek Collison <derek@nats.io>
derekcollison
force-pushed
the
interest-stresser
branch
from
3581673
to
ddfa5cd
Compare
Signed-off-by: Derek Collison <derek@nats.io>
… error Signed-off-by: Derek Collison <derek@nats.io>
neilalexander
approved these changes
Mar 29, 2023
…sed the lock Signed-off-by: Derek Collison <derek@nats.io>
Pre-allocate Co-authored-by: Neil <neil@nats.io>
Pre-allocate Co-authored-by: Neil <neil@nats.io>
Jarema
reviewed
Mar 29, 2023
Signed-off-by: Derek Collison <derek@nats.io>
…om underneath of us. Signed-off-by: Derek Collison <derek@nats.io>
Signed-off-by: Derek Collison <derek@nats.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR has general improvements and fixes to filestore, raft, and the clustering layer.
Summary
Signed-off-by: Derek Collison derek@nats.io