Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

com.hazelcast.jet.impl.processor.AsyncTransformUsingServiceP_IntegrationTest.stressTest_withRestart_graceful[ordered=true] #26333

Open
sumnerib opened this issue May 13, 2024 · 0 comments
Open

com.hazelcast.jet.impl.processor.AsyncTransformUsingServiceP_IntegrationTest.stressTest_withRestart_graceful[ordered=true] #26333

sumnerib opened this issue May 13, 2024 · 0 comments
Labels
Source: Internal PR or issue was opened by an employee Source: Jet Issues/PRs needed for Jet, but in other modules than Jet Team: Core Type: Test-Failure
Milestone
5.4.z

Comments

@sumnerib
Copy link
Contributor

5.4.z (commit 9191ef1b22e4a03e667c873f23f46e403db490d7)

Failed on openj9-21: https://jenkins.hazelcast.com/job/Hazelcast-5.maintenance-Openj9-21/2/testReport/com.hazelcast.jet.impl.processor/AsyncTransformUsingServiceP_IntegrationTest/stressTest_withRestart_graceful_ordered_true_/

Stacktrace: 
java.lang.NullPointerException
	at java.base/java.lang.StringConcatHelper.simpleConcat(StringConcatHelper.java)
	at com.hazelcast.jet.core.JetTestSupport.lambda$assertJobStatusEventually$2(JetTestSupport.java:338)
	at com.hazelcast.test.HazelcastTestSupport.assertTrueEventually(HazelcastTestSupport.java:1200)
	at com.hazelcast.test.HazelcastTestSupport.assertTrueEventually(HazelcastTestSupport.java:1219)
	at com.hazelcast.jet.core.JetTestSupport.assertJobStatusEventually(JetTestSupport.java:337)
	at com.hazelcast.jet.core.JetTestSupport.assertJobStatusEventually(JetTestSupport.java:253)
	at com.hazelcast.jet.impl.processor.AsyncTransformUsingServiceP_IntegrationTest.stressTestInt(AsyncTransformUsingServiceP_IntegrationTest.java:168)
	at com.hazelcast.jet.impl.processor.AsyncTransformUsingServiceP_IntegrationTest.stressTest_withRestart_graceful(AsyncTransformUsingServiceP_IntegrationTest.java:126)
	at java.base/java.lang.reflect.Method.invoke(Method.java:586)
	at com.hazelcast.test.FailOnTimeoutStatement$CallableStatement.call(FailOnTimeoutStatement.java:115)
	at com.hazelcast.test.FailOnTimeoutStatement$CallableStatement.call(FailOnTimeoutStatement.java:107)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.lang.Thread.run(Thread.java:1595)
Standard output: 
01:23:44,984  INFO || - [MetricsConfigHelper] Time-limited test - [LOCAL] [dev] [5.4.1-SNAPSHOT] Overridden metrics configuration with system property 'hazelcast.metrics.collection.frequency'='1' -> 'MetricsConfig.collectionFrequencySeconds'='1'
01:23:44,984  INFO || - [logo] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] 
	o    o     o     o---o   o--o o      o---o     o     o----o o--o--o
	|    |    / \       /         |     /         / \    |         |   
	o----o       o     o   o----o |    o             o   o----o    |   
	|    |  *     \   /           |     \       *     \       |    |   
	o    o *       o o---o   o--o o----o o---o *       o o----o    o   
01:23:44,984  INFO || - [system] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Copyright (c) 2008-2024, Hazelcast, Inc. All Rights Reserved.
01:23:44,984  INFO || - [system] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Hazelcast Platform 5.4.1-SNAPSHOT (20240513) starting at [127.0.0.1]:5701
01:23:44,984  INFO || - [system] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Cluster name: dev
01:23:44,984  INFO || - [system] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Integrity Checker is disabled. Fail-fast on corrupted executables will not be performed. For more information, see the documentation for Integrity Checker.
01:23:44,984  INFO || - [system] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Jet is enabled
01:23:44,986  INFO || - [MetricsConfigHelper] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Collecting debug metrics and sending to diagnostics is enabled
01:23:44,991  WARN || - [CPSubsystem] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
01:23:44,995  INFO || - [JetServiceBackend] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Setting number of cooperative threads and default parallelism to 2
01:23:44,996  INFO || - [Diagnostics] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
01:23:44,997  INFO || - [LifecycleService] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] [127.0.0.1]:5701 is STARTING
01:23:44,997  INFO || - [ClusterService] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] 

Members {size:1, ver:1} [
	Member [127.0.0.1]:5701 - 68ce921e-3234-498a-b9e3-1f75e69020d3 this
]

01:23:44,997  INFO || - [JobCoordinationService] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Jet started scanning for jobs
01:23:44,997  INFO || - [LifecycleService] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] [127.0.0.1]:5701 is STARTED
01:23:44,997 DEBUG || - [JobCoordinationService] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Not starting jobs because partitions are not yet initialized.
Started Running Test: stressTest_withRestart_graceful[ordered=true]
01:23:45,000  INFO |stressTest_withRestart_graceful[ordered=true]| - [PartitionStateManager] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Initializing cluster partition table arrangement...
01:23:45,074 DEBUG || - [JobClassLoaderService] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Creating job classLoader for job 0bb0-21f7-4c00-0001
01:23:45,074 DEBUG || - [JobClassLoaderService] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Create processor classloader map for job 0bb0-21f7-4c00-0001
01:23:45,081  INFO || - [JobCoordinationService] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Starting job 0bb0-21f7-4c00-0001 based on submit request
01:23:45,109  INFO || - [MasterJobContext] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Didn't find any snapshot to restore for job '0bb0-21f7-4c00-0001', execution 0bb0-21f7-4c01-0001
01:23:45,109  INFO || - [MasterJobContext] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Start executing job '0bb0-21f7-4c00-0001', execution 0bb0-21f7-4c01-0001, execution graph in DOT format:
digraph DAG {
	"source" [localParallelism=2];
	"map" [localParallelism=2];
	"sink" [localParallelism=1];
	"source" -> "map" [queueSize=128];
	"map" -> "sink" [queueSize=10];
}
HINT: You can use graphviz or http://viz-js.com to visualize the printed graph.
01:23:45,109 DEBUG || - [MasterJobContext] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Building execution plan for job '0bb0-21f7-4c00-0001', execution 0bb0-21f7-4c01-0001
01:23:45,113 DEBUG || - [JobRepository] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-1 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Job cleanup took 1ms
01:23:45,115 DEBUG || - [MasterJobContext] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Built execution plans for job '0bb0-21f7-4c00-0001', execution 0bb0-21f7-4c01-0001
01:23:45,118  INFO |stressTest_withRestart_graceful[ordered=true]| - [SimpleTestInClusterSupport] Time-limited test - Ditching 1 jobs in SimpleTestInClusterSupport.@After: [0bb0-21f7-4c00-0001]
01:23:45,118 DEBUG |stressTest_withRestart_graceful[ordered=true]| - [AbstractJobProxy] Time-limited test - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Sending CANCEL_FORCEFUL request for job 0bb0-21f7-4c00-0001 (name ??)
01:23:45,119 DEBUG || - [InitExecutionOperation] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Initializing execution plan for job 0bb0-21f7-4c00-0001, execution 0bb0-21f7-4c01-0001 from [127.0.0.1]:5701
01:23:45,122  INFO || - [JobExecutionService] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-1 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Execution plan for jobId=0bb0-21f7-4c00-0001, jobName='0bb0-21f7-4c00-0001', executionId=0bb0-21f7-4c01-0001 initialized
01:23:45,123 DEBUG || - [MasterJobContext] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] job '0bb0-21f7-4c00-0001', execution 0bb0-21f7-4c01-0001 to be cancelled after Init
01:23:45,123 DEBUG || - [JobExecutionService] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.async.thread-4 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] job '0bb0-21f7-4c00-0001', execution 0bb0-21f7-4c01-0001 calling completeExecution because execution terminated before it started
01:23:45,124 DEBUG || - [JobExecutionService] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.async.thread-4 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Completed execution of job '0bb0-21f7-4c00-0001', execution 0bb0-21f7-4c01-0001
01:23:45,125 DEBUG || - [JobClassLoaderService] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Finish JobClassLoaders phaseCount = 0, removing classloaders for jobId=0bb0-21f7-4c00-0001
01:23:45,125  INFO || - [MasterJobContext] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] Execution of job '0bb0-21f7-4c00-0001', execution 0bb0-21f7-4c01-0001 got terminated, reason=com.hazelcast.jet.impl.exception.CancellationByUserException
	Start time: 2024-05-13T01:23:45.081
	Duration: 00:00:00.044
	To see additional job metrics enable JobConfig.storeMetricsAfterJobCompletion
01:23:45,125 DEBUG || - [JobCoordinationService] hz.AsyncTransformUsingServiceP_IntegrationTest_condescending_goldstine.cached.thread-2 - [127.0.0.1]:5701 [dev] [5.4.1-SNAPSHOT] job '0bb0-21f7-4c00-0001', execution 0bb0-21f7-4c01-0001 is completed
01:23:45,128  INFO |stressTest_withRestart_graceful[ordered=true]| - [SimpleTestInClusterSupport] Time-limited test - Destroying 6 distributed objects in SimpleTestInClusterSupport.@After: [hz:impl:listService/sinkList1535e0df-0252-409b-9595-cda47cc1eb10, hz:impl:mapService/__jet.executionRecords, hz:impl:mapService/__jet.results, hz:impl:mapService/__jet.records, hz:impl:mapService/journaledMap1b842ece-2866-46fb-8946-6542abdcef64, hz:impl:flakeIdGeneratorService/__jet.ids]
01:23:45,137  INFO |stressTest_withRestart_graceful[ordered=true]| - [HazelcastTestSupport] Time-limited test - Shutting down node factory as @After action
BuildInfo right after stressTest_withRestart_graceful[ordered=true](com.hazelcast.jet.impl.processor.AsyncTransformUsingServiceP_IntegrationTest): BuildInfo{version='5.4.1-SNAPSHOT', build='20240513', buildNumber=20240513, revision=, enterprise=false, serializationVersion=1}
Hiccups measured while running test 'stressTest_withRestart_graceful[ordered=true](com.hazelcast.jet.impl.processor.AsyncTransformUsingServiceP_IntegrationTest):'
01:23:40, accumulated pauses: 33 ms, max pause: 1 ms, pauses over 1000 ms: 0
01:23:45, accumulated pauses: 18 ms, max pause: 17 ms, pauses over 1000 ms: 0


No metrics recorded during the test

Standard output can be found here - https://s3.console.aws.amazon.com/s3/buckets/j-artifacts/Hazelcast-5.maintenance-Openj9-21/2/
@sumnerib sumnerib added Team: Core Type: Test-Failure Source: Internal PR or issue was opened by an employee Source: Jet Issues/PRs needed for Jet, but in other modules than Jet labels May 13, 2024
@sumnerib sumnerib added this to the 5.4.z milestone May 13, 2024
@k-jamroz k-jamroz mentioned this issue May 14, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Source: Internal PR or issue was opened by an employee Source: Jet Issues/PRs needed for Jet, but in other modules than Jet Team: Core Type: Test-Failure
Projects
None yet
Milestone
5.4.z
Development

No branches or pull requests
1 participant
@sumnerib