Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[connector/failover] Test TestCurrentPipelineWithRetry fails intermittently #32396

Closed
crobert-1 opened this issue Apr 15, 2024 · 2 comments · Fixed by #32419
Closed

[connector/failover] Test TestCurrentPipelineWithRetry fails intermittently #32396

crobert-1 opened this issue Apr 15, 2024 · 2 comments · Fixed by #32419
Labels
connector/failover flaky test a test is flaky

Comments

@crobert-1
Copy link
Member

crobert-1 commented Apr 15, 2024 •
edited

Component(s)

connector/failover

Describe the issue you're reporting

CI/CD failure link

Failure output:

Running target 'test-with-cover' in module 'connector/failoverconnector' as part of group 'connector'
make --no-print-directory -C connector/failoverconnector test-with-cover
mkdir -p /home/runner/actions-runner/_work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/coverage/unit
/home/runner/actions-runner/_work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/.tools/gotestsum --rerun-fails=1 --packages="./..." -- -race -timeout 300s -parallel 4 --tags="" -cover -covermode=atomic -args -test.gocoverdir="/home/runner/actions-runner/_work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/coverage/unit"
∅  internal/metadata
✓  . (1.734s) (coverage: 86.4% of statements)
✖  internal/state (3.064s) (coverage: 78.8% of statements)

DONE 29 tests, 1 failure in 6.7[40](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/8694589952/job/23843888472#step:9:41)s

✓  internal/state (1.134s) (coverage: 78.8% of statements)

=== Failed
=== FAIL: internal/state TestCurrentPipelineWithRetry (3.00s)
    pipeline_selector_test.go:77: 
        	Error Trace:	/home/runner/actions-runner/_work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/connector/failoverconnector/internal/state/pipeline_selector_test.go:77
        	Error:      	Condition never satisfied
        	Test:       	TestCurrentPipelineWithRetry

DONE 2 runs, 30 tests, 1 failure in 8.[41](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/8694589952/job/23843888472#step:9:42)3s
@crobert-1 crobert-1 added the needs triage New item requiring triage label Apr 15, 2024
@github-actions GitHub Actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@crobert-1 crobert-1 added flaky test a test is flaky arm64 Issues related to arm64 architecture labels Apr 15, 2024
@crobert-1
Copy link
Member Author

+1 freq: https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/8697701081/job/23853491688?pr=32405

Looks like this is failing on Ubuntu-based runners as well, updating labels and title accordingly.

@crobert-1 crobert-1 removed the arm64 Issues related to arm64 architecture label Apr 16, 2024
@crobert-1 crobert-1 changed the title [connector/failover] Test TestCurrentPipelineWithRetry fails intermittently on actuated ARM runners [connector/failover] Test TestCurrentPipelineWithRetry fails intermittently Apr 16, 2024
@crobert-1 crobert-1 mentioned this issue Apr 16, 2024
Open
@github-actions github-actions bot mentioned this issue Apr 16, 2024
Open
@akats7 akats7 mentioned this issue Apr 16, 2024
Merged
djaglowski added a commit that referenced this issue Apr 16, 2024
@akats7 @djaglowski
[connector/failover] Fix flaky test (#32419)
ce92d2e 
**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
Update test structure to fix flaky behavior, originally seemed to fail
every 1-2k runs from my local testing.

**Link to tracking Issue:** <Issue number if applicable>

resolves #32396

**Testing:** <Describe what testing was performed and which tests were
added.>
Added test method to directly trigger retry for testing, Rerun 10k+
times

**Documentation:** <Describe the documentation added.>

---------

Co-authored-by: Daniel Jaglowski <jaglows3@gmail.com>
@djaglowski djaglowski removed the needs triage New item requiring triage label Apr 17, 2024
rimitchell pushed a commit to rimitchell/opentelemetry-collector-contrib that referenced this issue May 8, 2024
@akats7 @djaglowski @rimitchell
[connector/failover] Fix flaky test (open-telemetry#32419)
4ee4d0d 
**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
Update test structure to fix flaky behavior, originally seemed to fail
every 1-2k runs from my local testing.

**Link to tracking Issue:** <Issue number if applicable>

resolves open-telemetry#32396

**Testing:** <Describe what testing was performed and which tests were
added.>
Added test method to directly trigger retry for testing, Rerun 10k+
times

**Documentation:** <Describe the documentation added.>

---------

Co-authored-by: Daniel Jaglowski <jaglows3@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
connector/failover flaky test a test is flaky
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[connector/failover] Fix flaky test akats7/opentelemetry-collector-contrib
2 participants
@djaglowski @crobert-1