Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Race condition in make test-docker #127

Open
natebrennand opened this issue Apr 18, 2022 · 3 comments
Open

Race condition in make test-docker #127

natebrennand opened this issue Apr 18, 2022 · 3 comments
Labels
status: work in progress Twilio or the community is in the process of implementing type: bug bug in the library

Comments

@natebrennand
Copy link

Issue Summary

When running make test-docker locally, the first test executed by Go, TestGet, failed reliably.

When inspecting the logs from docker-compose, the Prism startup output was emitted long after the go tests executed and failed.
This makes me suspect that what I'm encountering is a startup issue where the Prism container isn't ready to accept requests before the go tests begin.
Mitigation suggestions are here: https://docs.docker.com/compose/startup-order/

Steps to Reproduce

  1. Run make test-docker
  2. Run docker-compose logs -f from examples/prism/ to see log output

Exception/Log

I added a log to better inspect the error occurring:

diff --git a/examples/go-client/helper/rest/api/v2010/api_test.go b/examples/go-client/helper/rest/api/v2010/api_test.go
index e75d78d..e56d02c 100644
--- a/examples/go-client/helper/rest/api/v2010/api_test.go
+++ b/examples/go-client/helper/rest/api/v2010/api_test.go
@@ -28,6 +28,7 @@ func TestMain(m *testing.M) {

 func TestGet(t *testing.T) {
        resp, err := testApiService.PageCredentialAws(nil, "", "")
+       t.Logf("err: %s", err)

Which then yielded this error:
prism-go-client-test-1 | {"Time":"2022-04-18T18:49:46.703409863Z","Action":"output","Package":"go-client/helper/rest/api/v2010","Test":"TestGet","Output":" api_test.go:31: err: Get \"http://prism_twilio:4010/v1/Credentials/AWS\": dial tcp 172.24.0.2:4010: connect: connection refused\n"}

Technical details:

  • twilio-go version: v0.24.0 (present main)
  • go version: 1.18
@childish-sambino
Copy link
Contributor

Hrm, I understand why this could be a problem, but I'm not seeing it locally. We also run these tests daily and not seeing any issues:

run: make test-docker

Is there some other piece that I'm missing which is causing the failure for you?

@childish-sambino childish-sambino added type: question question directed at the library status: waiting for feedback waiting for feedback from the submitter labels Apr 20, 2022
@natebrennand
Copy link
Author

Is there some other piece that I'm missing which is causing the failure for you?

My best guess is my local environment contributes to it? I've got a souped up M1 Max and 1gb fiber.

AIUI, this failure only occurs if examples/go-client/Dockerfile can execute those steps and start the tests before the prism container is ready to start requests.
The go-client test execution might never win the race in an environment with some combination of:

  • slower network access that rate limits the installation of the dependencies
  • limited compute resources to compile the Go libraries prior to executing tests

In this past execution, you can see that this race was narrowly lost, Prism starts up ~6s before the tests start executing:
https://github.com/twilio/twilio-oai-generator/runs/5561494883?check_suite_focus=true#step:6:10273
(I had to link to an older build because these logs aren't included in recent runs after the move to daemon mode for docker-compose)

@childish-sambino
Copy link
Contributor

Makes sense, will take a look at the PR.

@childish-sambino childish-sambino added type: bug bug in the library status: work in progress Twilio or the community is in the process of implementing and removed type: question question directed at the library status: waiting for feedback waiting for feedback from the submitter labels Apr 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: work in progress Twilio or the community is in the process of implementing type: bug bug in the library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants