Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As a product owner I want that message timestamping is refactored, so that it is more robust #1361

Open
2 tasks
raits opened this issue Sep 22, 2022 · 0 comments
Labels
help wanted Extra attention is needed

Comments

@raits
Copy link
Contributor

raits commented Sep 22, 2022

The fix created in XRDDEV-145 slightly increases the possibility that, when using batch timestamping, some messages are timestamped twice. This is inherently due to the implementation that can start a new batch timestamping process before the previous one is finished.

LogManager#TimeStamperJob schedules timestamping periodically

TaskQueue handles the time stamping requests, but the actual time stamping is handled asynchronously by Timestamper and temporary TimeStamperWorker, making it possible that several time stamping workers are running at the same time.

The results are written to the database only after the timestamping succeeds. Therefore, if a new job is started while the previous one is running, the new job will see the same records.

The JIRA issue this was created from can be found here: https://nordic-institute.atlassian.net/browse/XRDDEV-174

Acceptance criteria:

  • A new timestamping process should not start until the results of the previous batch are committed (Alternatively, concurrent timestamping processes should not process the same records).
  • The timestamping should adapt to load so that it does not fall behind (see https://nordic-institute.atlassian.net/browse/XRDDEV-145 for additional details).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
Status: Todo
Development

No branches or pull requests

1 participant