Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(spark): add all faulty servers into blacklist while writing for stage recomputing #1578

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

zuston
Copy link
Member

@zuston zuston commented Mar 14, 2024

What changes were proposed in this pull request?

add all faulty servers into blacklist while writing for stage recomputing

Why are the changes needed?

It will fail on retry when only one server is added into blacklist.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing tests.

@zuston
Copy link
Member Author

zuston commented Mar 14, 2024

Could you help review this? @yl09099

@codecov-commenter
Copy link

codecov-commenter commented Mar 14, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 54.95%. Comparing base (f6569b0) to head (1f1d00c).
Report is 5 commits behind head on master.

Files Patch % Lines
...fle/shuffle/manager/ShuffleManagerGrpcService.java 50.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #1578      +/-   ##
============================================
+ Coverage     53.99%   54.95%   +0.96%     
  Complexity     2862     2862              
============================================
  Files           438      418      -20     
  Lines         24819    22480    -2339     
  Branches       2111     2113       +2     
============================================
- Hits          13402    12355    -1047     
+ Misses        10578     9355    -1223     
+ Partials        839      770      -69     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Mar 14, 2024

Test Results

 2 315 files   - 25   2 315 suites   - 25   3h 57m 13s ⏱️ - 34m 43s
   892 tests  - 16     891 ✅  - 16   1 💤 ±0  0 ❌ ±0 
10 508 runs   - 33  10 494 ✅  - 33  14 💤 ±0  0 ❌ ±0 

Results for commit 1f1d00c. ± Comparison against base commit f6569b0.

This pull request removes 16 tests.
org.apache.uniffle.test.DynamicConfTest ‑ dynamicConfTest
org.apache.uniffle.test.HadoopConfTest ‑ hadoopConfTest
org.apache.uniffle.test.LargeSorterTest ‑ largeSorterTest
org.apache.uniffle.test.SecondarySortTest ‑ secondarySortTest
org.apache.uniffle.test.TezCartesianProductTest ‑ cartesianProductTest
org.apache.uniffle.test.TezHashJoinTest ‑ hashJoinDoBroadcastTest
org.apache.uniffle.test.TezHashJoinTest ‑ hashJoinTest
org.apache.uniffle.test.TezOrderedWordCountTest ‑ orderedWordCountTest
org.apache.uniffle.test.TezSimpleSessionExampleTest ‑ simpleSessionExampleTest
org.apache.uniffle.test.TezSortMergeJoinTest ‑ sortMergeJoinTest
…

♻️ This comment has been updated with latest results.

@yl09099
Copy link
Contributor

yl09099 commented Mar 17, 2024

LGTM~

@jerqi
Copy link
Contributor

jerqi commented Mar 18, 2024

Do you add a new ut for this case?

@zuston
Copy link
Member Author

zuston commented Mar 18, 2024

Do you add a new ut for this case?

I'm still reviewing this part.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants