Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests/random: futurize make_random_batches #18388

Merged
merged 4 commits into from May 14, 2024

Conversation

bharathv
Copy link
Contributor

@bharathv bharathv commented May 10, 2024

Yield periodically to ensure that when replicating large payloads (with lots of batches), reactor is not stalled. Adds some additional debug logging in raft fixture that attempts leader retries.

(probably) fixes #18329

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v24.1.x
  • v23.3.x
  • v23.2.x

Release Notes

  • none

@bharathv
Copy link
Contributor Author

/ci-repeat 5 debug

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented May 10, 2024

new failures in https://buildkite.com/redpanda/redpanda/builds/48914#018f60be-5152-456d-9a4e-885a0db547d2:

"rptest.tests.e2e_iam_role_test.AWSRoleFetchTests.test_write"

new failures in https://buildkite.com/redpanda/redpanda/builds/48914#018f60be-5149-41f1-a123-9f1b3a093f8b:

"rptest.tests.e2e_iam_role_test.ShortLivedCredentialsTests.test_short_lived_credentials"

new failures in https://buildkite.com/redpanda/redpanda/builds/49063#018f756c-19c9-48ef-ab60-b5c54e8e89b2:

"rptest.tests.cloud_storage_scrubber_test.CloudStorageScrubberTest.test_scrubber.cloud_storage_type=CloudStorageType.ABS"

@bharathv
Copy link
Contributor Author

/ci-repeat 1 debug

1 similar comment
@bharathv
Copy link
Contributor Author

/ci-repeat 1 debug

@bharathv
Copy link
Contributor Author

/ci-repeat 1

@bharathv bharathv changed the title dnm tests/random: futurize make_random_batches May 11, 2024
@bharathv bharathv marked this pull request as ready for review May 11, 2024 07:18
@@ -273,7 +274,7 @@ make_random_memory_record_batch_reader(record_batch_spec spec, int n_batches) {
if (n_batches--) {
auto batch_spec = spec;
batch_spec.offset = offset;
batches = make_random_batches(batch_spec);
batches = make_random_batches(batch_spec).get();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this should not use get() as it is not guaranteed that this will be called in the context of seastar thread

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like this is not used anywhere, I removed it for now.

Comment on lines +256 to +261
if (i % 5 == 0) {
co_await ss::coroutine::maybe_yield();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we also yield in the other overload of make_random_batches ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@bharathv
Copy link
Contributor Author

/ci-repeat 1

@bharathv
Copy link
Contributor Author

Failure unrelated #16474

@piyushredpanda piyushredpanda merged commit c55d8d6 into redpanda-data:dev May 14, 2024
14 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI Failure (key symptom) in test_raft_rpunit - "test_linarizable_barrier": critical check success has failed
4 participants