Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BACKPORT] Do not try to connect to the old member list after the cluster changes [API-2038] (#24745) #25504

Merged

Conversation

ihsandemir
Copy link
Contributor

Backports #24745

…s [API-2038] (hazelcast#24745)

During failover, when the client is about to try to connect to the next
cluster, it resets some services including the cluster service.

In its reset logic, cluster service clears the member list version, but
leaves the members as it is, as there are some services/proxies that
rely on the fact that the member list cannot be empty.

This creates a problem for the failover case. After the cluster change,
the client still tries to connect to the last known member list, which
is the members of the previous cluster.

To solve the problem, I have introduced a new method to the cluster
service, which returns the "effective" member list. That is, it returns
an empty list after the service is reset, but returns the member list as
expected otherwise. With that, the connection logic simply skips the old
member list after the client decides to change the cluster to try to
connect to.
@ihsandemir ihsandemir added this to the 5.3.3 milestone Sep 21, 2023
@ihsandemir ihsandemir requested a review from a team as a code owner September 21, 2023 09:39
@ihsandemir ihsandemir self-assigned this Sep 21, 2023
@hz-devops-test
Copy link

The job Hazelcast-pr-EE-compiler of your PR failed. (Hazelcast internal details: build log, artifacts).
Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log file
---------ERRORS-----------
--------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:license-maven-plugin:2.0.1:add-third-party (add-third-party) on project hazelcast-jet-files-gcs: could not init goal AddThirdPartyMojo for reason : null: ConcurrentModificationException -> [Help 1]
--------------------------
[ERROR] 
--------------------------
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
--------------------------
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
--------------------------
[ERROR] 
--------------------------
[ERROR] For more information about the errors and possible solutions, please read the following articles:
--------------------------
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
--------------------------
[ERROR] 
--------------------------
[ERROR] After correcting the problems, you can resume the build with the command
--------------------------
[ERROR]   mvn  -rf :hazelcast-jet-files-gcs
--------------------------

Copy link
Member

@srknzl srknzl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like a 1-1 backport. ✅

@ihsandemir
Copy link
Contributor Author

run-lab-run

@ihsandemir
Copy link
Contributor Author

run-ee-tests

@ihsandemir
Copy link
Contributor Author

run-ee-pr

@hz-devops-test
Copy link

The job Hazelcast-pr-builder-ee-tests of your PR failed. (Hazelcast internal details: build log, artifacts).
Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log file
--------------------------
---------SUMMARY----------
--------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle (default) on project hazelcast-enterprise-root: Execution default of goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle failed: An API incompatibility was encountered while executing org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle: java.lang.UnsupportedClassVersionError: com/puppycrawl/tools/checkstyle/api/CheckstyleException has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
--------------------------
---------ERRORS-----------
--------------------------
[ERROR] 'dependencies.dependency.groupId' for $com.google.protobuf:protobuf-java:jar with value '$com.google.protobuf' does not match a valid id pattern. @ line 17, column 16
--------------------------
[ERROR] 'dependencies.dependency.version' for $com.google.protobuf:protobuf-java:jar is missing. @ line 16, column 17
--------------------------
[ERROR] 'dependencies.dependency.groupId' for $com.google.protobuf:protobuf-java:jar with value '$com.google.protobuf' does not match a valid id pattern. @ line 17, column 16
--------------------------
[ERROR] 'dependencies.dependency.version' for $com.google.protobuf:protobuf-java:jar is missing. @ line 16, column 17
--------------------------
[ERROR] 'dependencies.dependency.groupId' for $com.google.protobuf:protobuf-java:jar with value '$com.google.protobuf' does not match a valid id pattern. @ line 17, column 16
--------------------------
[ERROR] 'dependencies.dependency.version' for $com.google.protobuf:protobuf-java:jar is missing. @ line 16, column 17
--------------------------
[ERROR] 'dependencies.dependency.groupId' for $com.google.protobuf:protobuf-java:jar with value '$com.google.protobuf' does not match a valid id pattern. @ line 17, column 16
--------------------------
[ERROR] 'dependencies.dependency.version' for $com.google.protobuf:protobuf-java:jar is missing. @ line 16, column 17
--------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle (default) on project hazelcast-enterprise-root: Execution default of goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle failed: An API incompatibility was encountered while executing org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:checkstyle: java.lang.UnsupportedClassVersionError: com/puppycrawl/tools/checkstyle/api/CheckstyleException has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
--------------------------
[ERROR] -----------------------------------------------------
--------------------------
[ERROR] realm =    plugin>org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0
--------------------------
[ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
--------------------------
[ERROR] urls[0] = file:/home/jenkins/.m2/repository/org/apache/maven/plugins/maven-checkstyle-plugin/3.3.0/maven-checkstyle-plugin-3.3.0.jar
--------------------------
[ERROR] urls[1] = file:/home/jenkins/.m2/repository/com/puppycrawl/tools/checkstyle/10.12.2/checkstyle-10.12.2.jar
--------------------------
[ERROR] urls[2] = file:/home/jenkins/.m2/repository/info/picocli/picocli/4.7.4/picocli-4.7.4.jar
--------------------------
[ERROR] urls[3] = file:/home/jenkins/.m2/repository/org/antlr/antlr4-runtime/4.11.1/antlr4-runtime-4.11.1.jar
--------------------------
[ERROR] urls[4] = file:/home/jenkins/.m2/repository/commons-beanutils/commons-beanutils/1.9.4/commons-beanutils-1.9.4.jar
--------------------------
[ERROR] urls[5] = file:/home/jenkins/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar
--------------------------
[ERROR] urls[6] = file:/home/jenkins/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar
--------------------------
[ERROR] urls[7] = file:/home/jenkins/.m2/repository/com/google/guava/guava/32.0.1-jre/guava-32.0.1-jre.jar
--------------------------
[ERROR] urls[8] = file:/home/jenkins/.m2/repository/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar
--------------------------
[ERROR] urls[9] = file:/home/jenkins/.m2/repository/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
--------------------------
[ERROR] urls[10] = file:/home/jenkins/.m2/repository/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar
--------------------------
[ERROR] urls[11] = file:/home/jenkins/.m2/repository/com/google/errorprone/error_prone_annotations/2.18.0/error_prone_annotations-2.18.0.jar
--------------------------
[ERROR] urls[12] = file:/home/jenkins/.m2/repository/com/google/j2objc/j2objc-annotations/2.8/j2objc-annotations-2.8.jar
--------------------------
[ERROR] urls[13] = file:/home/jenkins/.m2/repository/org/reflections/reflections/0.10.2/reflections-0.10.2.jar
--------------------------
[ERROR] urls[14] = file:/home/jenkins/.m2/repository/org/javassist/javassist/3.28.0-GA/javassist-3.28.0-GA.jar
--------------------------
[ERROR] urls[15] = file:/home/jenkins/.m2/repository/net/sf/saxon/Saxon-HE/12.3/Saxon-HE-12.3.jar
--------------------------
[ERROR] urls[16] = file:/home/jenkins/.m2/repository/org/xmlresolver/xmlresolver/5.2.0/xmlresolver-5.2.0.jar
--------------------------
[ERROR] urls[17] = file:/home/jenkins/.m2/repository/org/apache/httpcomponents/client5/httpclient5/5.1.3/httpclient5-5.1.3.jar
--------------------------
[ERROR] urls[18] = file:/home/jenkins/.m2/repository/org/apache/httpcomponents/core5/httpcore5-h2/5.1.3/httpcore5-h2-5.1.3.jar
--------------------------
[ERROR] urls[19] = file:/home/jenkins/.m2/repository/org/apache/httpcomponents/core5/httpcore5/5.1.3/httpcore5-5.1.3.jar
--------------------------
[ERROR] urls[20] = file:/home/jenkins/.m2/repository/org/xmlresolver/xmlresolver/5.2.0/xmlresolver-5.2.0-data.jar
--------------------------
[ERROR] urls[21] = file:/home/jenkins/.m2/repository/org/checkerframework/checker-qual/3.27.0/checker-qual-3.27.0.jar
--------------------------
[ERROR] urls[22] = file:/home/jenkins/.m2/repository/org/apache/maven/doxia/doxia-module-xdoc/1.12.0/doxia-module-xdoc-1.12.0.jar
--------------------------
[ERROR] urls[23] = file:/home/jenkins/.m2/repository/org/apache/maven/reporting/maven-reporting-api/3.1.1/maven-reporting-api-3.1.1.jar
--------------------------
[ERROR] urls[24] = file:/home/jenkins/.m2/repository/org/apache/maven/reporting/maven-reporting-impl/3.2.0/maven-reporting-impl-3.2.0.jar
--------------------------
[ERROR] urls[25] = file:/home/jenkins/.m2/repository/javax/enterprise/cdi-api/1.2/cdi-api-1.2.jar
--------------------------
[ERROR] urls[26] = file:/home/jenkins/.m2/repository/org/eclipse/sisu/org.eclipse.sisu.inject/0.3.5/org.eclipse.sisu.inject-0.3.5.jar
--------------------------
[ERROR] urls[27] = file:/home/jenkins/.m2/repository/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar
--------------------------
[ERROR] urls[28] = file:/home/jenkins/.m2/repository/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.jar
--------------------------
[ERROR] urls[29] = file:/home/jenkins/.m2/repository/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar
--------------------------
[ERROR] urls[30] = file:/home/jenkins/.m2/repository/org/apache/maven/doxia/doxia-decoration-model/1.11.1/doxia-decoration-model-1.11.1.jar
--------------------------
[ERROR] urls[31] = file:/home/jenkins/.m2/repository/org/apache/maven/doxia/doxia-site-renderer/1.11.1/doxia-site-renderer-1.11.1.jar
--------------------------
[ERROR] urls[32] = file:/home/jenkins/.m2/repository/org/apache/maven/doxia/doxia-skin-model/1.11.1/doxia-skin-model-1.11.1.jar
--------------------------
[ERROR] urls[33] = file:/home/jenkins/.m2/repository/org/apache/maven/doxia/doxia-module-xhtml/1.11.1/doxia-module-xhtml-1.11.1.jar
--------------------------
[ERROR] urls[34] = file:/home/jenkins/.m2/repository/org/apache/maven/doxia/doxia-module-xhtml5/1.11.1/doxia-module-xhtml5-1.11.1.jar
--------------------------
[ERROR] urls[35] = file:/home/jenkins/.m2/repository/org/codehaus/plexus/plexus-velocity/1.2/plexus-velocity-1.2.jar
--------------------------
[ERROR] urls[36] = file:/home/jenkins/.m2/repository/org/apache/velocity/velocity/1.7/velocity-1.7.jar
--------------------------
[ERROR] urls[37] = file:/home/jenkins/.m2/repository/commons-lang/commons-lang/2.4/commons-lang-2.4.jar
--------------------------
[ERROR] urls[38] = file:/home/jenkins/.m2/repository/org/apache/velocity/velocity-tools/2.0/velocity-tools-2.0.jar
--------------------------
[ERROR] urls[39] = file:/home/jenkins/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar
--------------------------
[ERROR] urls[40] = file:/home/jenkins/.m2/repository/commons-chain/commons-chain/1.1/commons-chain-1.1.jar
--------------------------
[ERROR] urls[41] = file:/home/jenkins/.m2/repository/dom4j/dom4j/1.1/dom4j-1.1.jar
--------------------------
[ERROR] urls[42] = file:/home/jenkins/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar
--------------------------
[ERROR] urls[43] = file:/home/jenkins/.m2/repository/org/apache/maven/doxia/doxia-sink-api/1.11.1/doxia-sink-api-1.11.1.jar
--------------------------
[ERROR] urls[44] = file:/home/jenkins/.m2/repository/org/apache/maven/doxia/doxia-logging-api/1.11.1/doxia-logging-api-1.11.1.jar
--------------------------
[ERROR] urls[45] = file:/home/jenkins/.m2/repository/org/apache/maven/doxia/doxia-core/1.11.1/doxia-core-1.11.1.jar
--------------------------
[ERROR] urls[46] = file:/home/jenkins/.m2/repository/org/apache/xbean/xbean-reflect/3.7/xbean-reflect-3.7.jar
--------------------------
[ERROR] urls[47] = file:/home/jenkins/.m2/repository/com/google/collections/google-collections/1.0/google-collections-1.0.jar
--------------------------
[ERROR] urls[48] = file:/home/jenkins/.m2/repository/org/apache/commons/commons-text/1.3/commons-text-1.3.jar
--------------------------
[ERROR] urls[49] = file:/home/jenkins/.m2/repository/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar
--------------------------
[ERROR] urls[50] = file:/home/jenkins/.m2/repository/commons-codec/commons-codec/1.11/commons-codec-1.11.jar
--------------------------
[ERROR] urls[51] = file:/home/jenkins/.m2/repository/org/apache/httpcomponents/httpcore/4.4.14/httpcore-4.4.14.jar
--------------------------
[ERROR] urls[52] = file:/home/jenkins/.m2/repository/org/apache/maven/doxia/doxia-integration-tools/1.11.1/doxia-integration-tools-1.11.1.jar
--------------------------
[ERROR] urls[53] = file:/home/jenkins/.m2/repository/commons-io/commons-io/2.6/commons-io-2.6.jar
--------------------------
[ERROR] urls[54] = file:/home/jenkins/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar
--------------------------
[ERROR] urls[55] = file:/home/jenkins/.m2/repository/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.jar
--------------------------
[ERROR] urls[56] = file:/home/jenkins/.m2/repository/org/codehaus/plexus/plexus-resources/1.1.0/plexus-resources-1.1.0.jar
--------------------------
[ERROR] urls[57] = file:/home/jenkins/.m2/repository/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar
--------------------------
[ERROR] urls[58] = file:/home/jenkins/.m2/repository/org/codehaus/plexus/plexus-i18n/1.0-beta-10/plexus-i18n-1.0-beta-10.jar
--------------------------
[ERROR] urls[59] = file:/home/jenkins/.m2/repository/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar
--------------------------
[ERROR] Number of foreign imports: 1
--------------------------
[ERROR] import: Entry[import  from realm ClassRealm[project>com.hazelcast:hazelcast-enterprise-root:5.4.0-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]]
--------------------------
[ERROR] 
--------------------------
[ERROR] -----------------------------------------------------
--------------------------
[ERROR] -> [Help 1]
--------------------------
[ERROR] 
--------------------------
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
--------------------------
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
--------------------------
[ERROR] 
--------------------------
[ERROR] For more information about the errors and possible solutions, please read the following articles:
--------------------------
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginContainerException
--------------------------

@ihsandemir ihsandemir merged commit 4fb8074 into hazelcast:5.3.z Sep 27, 2023
7 of 9 checks passed
@ihsandemir ihsandemir deleted the backports/5.3.z/failover-old-memberlist branch September 27, 2023 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants