Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: uses ThreadLocal to prevent concurrent modification on ScopeManager #5617

Conversation

filipelautert
Copy link
Collaborator

@filipelautert filipelautert commented Feb 21, 2024

Impact

  • Bug fix (non-breaking change which fixes expected existing functionality)
  • Enhancement/New feature (adds functionality without impacting existing logic)
  • Breaking change (fix or feature that would cause existing functionality to change)

Description

Issues #5426 #2433 #4173 #4314 presents the same error: "Cannot end scope bsbacsmwja when currently at scope root" .
The issue was narrowed down to Scope class handling of ScopeManager:

  1. Thread "A" starts and creates a root scope. By doing so, Scope.getCurrentScope() returns "root"
  2. Thread "A" starts a child scope "ABC". Now, Scope.getCurrentScope() returns "ABC", and parent is root (parent is saved in a non-static variable, while currentScope is static)
  3. Thread "A" is suspended and Thread "B" starts
  4. Thread "B" starts. By doing so, Scope.getCurrentScope() returns "ABC" (hey! that's already wrong)
  5. Thread "B" starts a child scope "DEF". Now, Scope.getCurrentScope() returns "DEF" and parent is "ABC"
  6. Thread "B" is suspended, "A" resumes
  7. Thread "A" does it's job and ends the scope. The last step of exit is restoring the former Scope, so thread "A" sets currentScope to root (remember, parent is local and current is static) .
  8. Thread "B" resumes - now working on curentScope "root" . It does it work and tries to finish. It tries to compare scopeId that was saved in a local variable, but then "B" gets message "Cannot end scope DEF (local variable) when currently at scope root (statically changed) " - RuntimeException

We prevent this issue by using a ThreadLocal object that ensures that each thread will have it's own ScopeManager , so "A" won't break "B" as it cannot mess with it's ScopeManager.

This issue has always been around, but seems it started to show up more as child scopes are being used for blocking tasks (that allows threads switching) such as database execution in ChangelogJdbcMdcListener . In the example above, A and B were suspended waiting for a database query to run.

@filipelautert filipelautert force-pushed the 5426-multiple-schema-upgrade-failed-when-done-through-multithreading branch from 0977c36 to a4874ea Compare February 27, 2024 10:31
Copy link
Contributor

@StevenMassaro StevenMassaro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job identifying the root cause here.

Is it possible to write a unit test for this? Seems like the steps you provided in the description should be pretty straightforward to write a test for.

Copy link
Contributor

@rberezen rberezen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

functional test failure is expected and will be fixed with merging new s3 extension

@filipelautert filipelautert merged commit d6316a0 into master Feb 29, 2024
32 of 34 checks passed
@filipelautert filipelautert deleted the 5426-multiple-schema-upgrade-failed-when-done-through-multithreading branch February 29, 2024 17:36
@filipelautert filipelautert added this to the 1NEXT milestone Mar 4, 2024
@mches
Copy link
Contributor

mches commented Mar 28, 2024

@filipelautert Nice fix. Would you suppose this makes ThreadLocalScopeManager effectively obsolete? Perhaps it should be deprecated?

@filipelautert
Copy link
Collaborator Author

@mches makes sense as now we are always using it, and in a simpler way.

@filipelautert
Copy link
Collaborator Author

@mches #5780 , thanks for the reminder!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Multiple schema upgrade failed when done through multithreading
5 participants