Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Attempt to fix some flaky Elixir unit tests #4927

Closed
wants to merge 4 commits into from

Conversation

jamilbk
Copy link
Member

@jamilbk jamilbk commented May 8, 2024

Possibly fixes some of the flaky unit tests that crop up from time to time.

Refs #4635

Copy link

vercel bot commented May 8, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
firezone ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 9, 2024 6:35pm

@github-actions github-actions bot added the kind/chore Issues related to repository cleanup or maintenance label May 8, 2024
Copy link

github-actions bot commented May 8, 2024

Terraform Cloud Plan Output

Plan: 15 to add, 15 to change, 15 to destroy.

Terraform Cloud Plan

@jamilbk jamilbk changed the title chore(ci): Attempt to mitigate some flaky Elixir unit tests ci: Attempt to mitigate some flaky Elixir unit tests May 8, 2024
@jamilbk jamilbk changed the title ci: Attempt to mitigate some flaky Elixir unit tests ci: Attempt to fix some flaky Elixir unit tests May 8, 2024
Copy link

github-actions bot commented May 8, 2024

Performance Test Results

TCP

Test Name Received/s Sent/s Retransmits
direct-tcp-client2server 238.7 MiB (-1%) 241.0 MiB (-0%) 352 (+3%)
direct-tcp-server2client 241.9 MiB (+0%) 243.6 MiB (+0%) 72 (-52%)
relayed-tcp-client2server 221.6 MiB (+1%) 222.5 MiB (+1%) 267 (-12%)
relayed-tcp-server2client 234.7 MiB (-0%) 235.3 MiB (-0%) 401 (-4%)

UDP

Test Name Total/s Jitter Lost
direct-udp-client2server 500.0 MiB (+0%) 0.09ms (+140%) 45.60% (+15%)
direct-udp-server2client 499.9 MiB (-0%) 0.01ms (-49%) 22.35% (+2%)
relayed-udp-client2server 500.0 MiB (-0%) 0.05ms (+9%) 56.63% (+2%)
relayed-udp-server2client 499.6 MiB (-0%) 0.03ms (+1435%) 34.59% (-16%)

@jamilbk jamilbk requested a review from AndrewDryga May 9, 2024 17:45
@@ -91,7 +90,13 @@ defmodule Domain.Mocks.OpenIDConnect do
"request_parameter_supported" => false
}

Plug.Conn.resp(conn, 200, Jason.encode!(attrs))
# Process may not be alive in slow CI environments
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -134,7 +134,8 @@ defmodule Web.AcceptanceCase do
if env = System.get_env("E2E_DEFAULT_WAIT_SECONDS") do
String.to_integer(env)
else
2
# GitHub shared runners can be very slow during peak hours
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jamilbk it's already 5 seconds on CI: E2E_DEFAULT_WAIT_SECONDS in

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😢

@github-actions github-actions bot added the area/ci Changes to the CI pipeline / Github Actions label May 9, 2024
Plug.Conn.resp(conn, 200, Jason.encode!(attrs))

# Process may not be alive in slow CI environments
if Process.alive?(test_pid) do
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't fix anything, the issue happens when Plug process is trying to read from Bypass process. So when the Plug process is terminated with the test process, the bypass can't finish reading the full response. And it doesn't matter if the response code is 503 or 401, has body or not, etc.

@jamilbk jamilbk closed this May 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ci Changes to the CI pipeline / Github Actions kind/chore Issues related to repository cleanup or maintenance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants