Flaky-test: pulsar.metadata.MetadataStoreTest.concurrentPutTest

[khac]

Search before asking

• I searched in the issues and found nothing similar.

Example failure

.

Exception stacktrace

I ran the code on local machine Mac M1 chip,

[ERROR] Tests run: 263, Failures: 1, Errors: 0, Skipped: 259, Time elapsed: 0.699 s <<< FAILURE! - in 
org.apache.pulsar.metadata.MetadataStoreTest
[ERROR] org.apache.pulsar.metadata.MetadataStoreTest.concurrentPutTest[Etcd, 
org.apache.pulsar.metadata.BaseMetadataStoreTest$$Lambda$923/0x000000c801725188@62a5e4c7](10)  
Time elapsed: 0.003 s  <<< FAILURE!
java.lang.IllegalStateException: Previous attempts to find a Docker environment failed. Will not retry. Please see logs and check configuration
        at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:231)
        at org.testcontainers.DockerClientFactory.getOrInitializeStrategy(DockerClientFactory.java:150)
        at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:191)

Are you willing to submit a PR?

• I'm willing to submit a PR!

[lhotari]

We do have a lot of flaky tests. However in this case, the error message "java.lang.IllegalStateException: Previous attempts to find a Docker environment failed. Will not retry. Please see logs and check configuration" doesn't seem to be about flakiness.

Please verify that you have a recent version of Docker installed. I believe that on Mac M1 that you'll need Docker Desktop. It's free for personal use. TestContainers Desktop is another alternative.

[khac]

When I run this test on debugger mode, it runs fine, when I run all tests this one fails metadata.MetadataStoreTest fail.

[ERROR] Failures: 
[ERROR]   MetadataStoreTest.testThreadSwitchOfZkMetadataStore:484 » Completion java.lang.AssertionError: 
Expect to switch to thread 196d8b0fd39848358a5ec527a7a34e48, but currently it is thread 
TestNG-method=testThreadSwitchOfZkMetadataStore-1 expected [true] but found [false]
[INFO] 
[ERROR] Tests run: 910, Failures: 1, Errors: 0, Skipped: 742

[lhotari]

When I run this test on debugger mode, it runs fine, when I run all tests this one fails metadata.MetadataStoreTest fail.

We have a lot of flaky tests.
For development, it's hopeless to run all tests locally. A better approach is to setup a "personal CI" where you run CI jobs in your GitHub fork of apache/pulsar. We have some docs for setting that up: https://pulsar.apache.org/contribute/personal-ci/ .
The guide doesn't mention that you have to enable GitHub Actions in your fork. It's no longer enabled by default.