Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: Change cloud regions #30378

Merged
merged 1 commit into from Feb 1, 2024
Merged

CI: Change cloud regions #30378

merged 1 commit into from Feb 1, 2024

Conversation

brlbil
Copy link
Contributor

@brlbil brlbil commented Jan 23, 2024 •
edited

Scheduled workflows run all the supported k8s versions on different regions. During some flake investigation, I have realized that regions in or close to the US take less time than others. Due to long-running tests, some workflows fail.
This PR changes regions to ones in the US or Canada. Run times before and after are down below.

Scheduled tests are simulated and links to the successful runs are below.
aks
aws-cni
eks
external-workloads
gke

Note: GKE runs 4 tests per k8s version, they all have similar run times, for simplicity only one of them was included.

As a side effect, Fixes: #30312

Average run times before the change

Workflow: Conformance AKS (ci-aks)
Job: Installation and Connectivity Test (1.25, westeurope, 1),  AvgRunTime: 47m9.875s
Job: Installation and Connectivity Test (1.26, westus, 2),  AvgRunTime: 42m3.5s
Job: Installation and Connectivity Test (1.27, eastasia, 3),  AvgRunTime: 59m46s
Job: Installation and Connectivity Test (1.28, eastus, 4, true),  AvgRunTime: 37m5.25s

Workflow: Conformance AWS-CNI (ci-awscni)
Job: Installation and Connectivity Test (1.23, eu-central-1),  AvgRunTime: 33m46.125s
Job: Installation and Connectivity Test (1.24, eu-west-1),  AvgRunTime: 56.3125s
Job: Installation and Connectivity Test (1.25, ap-northeast-1),  AvgRunTime: 33m45.5s
Job: Installation and Connectivity Test (1.26, us-east-2),  AvgRunTime: 25m59.0625s
Job: Installation and Connectivity Test (1.27, ca-central-1),  AvgRunTime: 25m24.125s
Job: Installation and Connectivity Test (1.28, eu-north-1, true),  AvgRunTime: 33m30.4375s

Workflow: Conformance External Workloads (ci-external-workloads)
Job: Installation and Connectivity Test (1.24, europe-west6-b, 1),  AvgRunTime: 30m22s
Job: Installation and Connectivity Test (1.25, us-west2-a, 2),  AvgRunTime: 25m57.6s
Job: Installation and Connectivity Test (1.26, asia-northeast1-c, 3),  AvgRunTime: 38m2.066666666s
Job: Installation and Connectivity Test (1.27, europe-north1-b, 4),  AvgRunTime: 34m17.933333333s
Job: Installation and Connectivity Test (1.28, us-east5-a, 5, true),  AvgRunTime: 24m27.933333333s

Workflow: Conformance EKS (ci-eks)
Job: Installation and Connectivity Test (1.23, eu-central-1),  AvgRunTime: 1h2m58.125s
Job: Installation and Connectivity Test (1.24, eu-west-1),  AvgRunTime: 55m31.375s
Job: Installation and Connectivity Test (1.25, ap-northeast-1),  AvgRunTime: 1h4m43.25s
Job: Installation and Connectivity Test (1.26, us-east-2),  AvgRunTime: 47m37.4375s
Job: Installation and Connectivity Test (1.27, ca-central-1),  AvgRunTime: 45m27.375s
Job: Installation and Connectivity Test (1.28, eu-north-1, true),  AvgRunTime: 57m59.25s

Workflow: Conformance GKE (ci-gke)
Job: Installation and Connectivity Test (1.24, europe-west6-b, 1, no-tunnel, 1),  AvgRunTime: 29m
Job: Installation and Connectivity Test (1.25, us-west2-a, 2, no-tunnel, 1),  AvgRunTime: 24m
Job: Installation and Connectivity Test (1.26, asia-northeast1-c, 3, no-tunnel, 1),  AvgRunTime: 36m
Job: Installation and Connectivity Test (1.27, europe-north1-b, 4, no-tunnel, 1),  AvgRunTime: 32m35.0625s
Job: Installation and Connectivity Test (1.28, us-east5-a, 5, true, no-tunnel, 1),  AvgRunTime: 23m30.3125s

Average run times after the change

Workflow: Conformance AKS (ci-aks)
Job: Installation and Connectivity Test (1.25, westus3, 1),  AvgRunTime: 41m22s
Job: Installation and Connectivity Test (1.26, westus2, 2),  AvgRunTime: 45m29s
Job: Installation and Connectivity Test (1.27, eastus2, 3),  AvgRunTime: 34m17s
Job: Installation and Connectivity Test (1.28, eastus, 4, true),  AvgRunTime: 38m33s

Workflow: Conformance AWS-CNI (ci-awscni)
Job: Installation and Connectivity Test (1.23, ca-west-1),  AvgRunTime: 27m37s
Job: Installation and Connectivity Test (1.24, us-west-2),  AvgRunTime: 24m34s
Job: Installation and Connectivity Test (1.25, us-west-1),  AvgRunTime: 27m28s
Job: Installation and Connectivity Test (1.26, us-east-2),  AvgRunTime: 25m46s
Job: Installation and Connectivity Test (1.27, ca-central-1),  AvgRunTime: 25m6s
Job: Installation and Connectivity Test (1.28, us-east-1, true),  AvgRunTime: 24m56s

Workflow: Conformance External Workloads (ci-external-workloads)
Job: Installation and Connectivity Test (1.24, us-west1-b, 1),  AvgRunTime: 27m23s
Job: Installation and Connectivity Test (1.25, us-west2-c, 2),  AvgRunTime: 27m45s
Job: Installation and Connectivity Test (1.26, us-west3-a, 3),  AvgRunTime: 29m2s
Job: Installation and Connectivity Test (1.27, us-east4-b, 4),  AvgRunTime: 23m3s
Job: Installation and Connectivity Test (1.28, us-east1-c, 5, true),  AvgRunTime: 25m7s

Workflow: Conformance EKS (ci-eks)
Job: Installation and Connectivity Test (1.23, ca-west-1),  AvgRunTime: 58m15s
Job: Installation and Connectivity Test (1.24, us-west-2),  AvgRunTime: 56m45s
Job: Installation and Connectivity Test (1.25, us-west-1),  AvgRunTime: 45m0s
Job: Installation and Connectivity Test (1.26, us-east-2),  AvgRunTime: 43m21s
Job: Installation and Connectivity Test (1.27, ca-central-1),  AvgRunTime: 43m42s
Job: Installation and Connectivity Test (1.28, us-east-1, true),  AvgRunTime: 51m28s

Workflow: Conformance GKE (ci-gke)
Job: Installation and Connectivity Test (1.24, us-west1-b, 1, no-tunnel, 1),  AvgRunTime: 23m34s
Job: Installation and Connectivity Test (1.25, us-west2-c, 2, no-tunnel, 1),  AvgRunTime: 22m2s
Job: Installation and Connectivity Test (1.26, us-west3-a, 3, no-tunnel, 1),  AvgRunTime: 24m47s
Job: Installation and Connectivity Test (1.27, us-east4-b, 4, no-tunnel, 1),  AvgRunTime: 25m32s
Job: Installation and Connectivity Test (1.28, us-east1-c, 5, true, no-tunnel, 1),  AvgRunTime: 24m19s

@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Jan 23, 2024
@brlbil brlbil force-pushed the pr/brlbil/change-ci-regions branch from 52ff552 to 5527603 Compare January 23, 2024 12:48
@brlbil brlbil added area/CI Continuous Integration testing issue or flake area/CI-improvement Topic or proposal to improve the Continuous Integration workflow labels Jan 23, 2024
@brlbil
Copy link
Contributor Author

brlbil commented Jan 23, 2024

/ci-aks

@brlbil
Copy link
Contributor Author

brlbil commented Jan 23, 2024

/ci-awscni

@brlbil
Copy link
Contributor Author

brlbil commented Jan 23, 2024

/ci-eks

@brlbil
Copy link
Contributor Author

brlbil commented Jan 23, 2024

/ci-external-workloads

@brlbil
Copy link
Contributor Author

brlbil commented Jan 23, 2024

/ci-gke

@brlbil
Copy link
Contributor Author

brlbil commented Jan 23, 2024

/ci-aks

@brlbil
Copy link
Contributor Author

brlbil commented Jan 23, 2024

/ci-awscni

@brlbil
Copy link
Contributor Author

brlbil commented Jan 23, 2024

/ci-eks

@brlbil
Copy link
Contributor Author

brlbil commented Jan 23, 2024

/ci-external-workloads

@brlbil
Copy link
Contributor Author

brlbil commented Jan 23, 2024

/ci-gke

@brlbil
Copy link
Contributor Author

brlbil commented Jan 24, 2024

/ci-aks

@brlbil
Copy link
Contributor Author

brlbil commented Jan 24, 2024

/ci-awscni

@brlbil
Copy link
Contributor Author

brlbil commented Jan 24, 2024

/ci-eks

@brlbil
Copy link
Contributor Author

brlbil commented Jan 24, 2024

/ci-external-workloads

@brlbil
Copy link
Contributor Author

brlbil commented Jan 24, 2024

/ci-gke

@brlbil brlbil force-pushed the pr/brlbil/change-ci-regions branch from 5527603 to e1d3299 Compare January 26, 2024 12:32
@brlbil
Copy link
Contributor Author

brlbil commented Jan 26, 2024

/ci-awscni

@brlbil
Copy link
Contributor Author

brlbil commented Jan 26, 2024

/ci-aks

@brlbil
Copy link
Contributor Author

brlbil commented Jan 26, 2024

/ci-gke

@brlbil
Copy link
Contributor Author

brlbil commented Jan 26, 2024

/ci-eks

@brlbil
Copy link
Contributor Author

brlbil commented Jan 26, 2024

/ci-external-workloads

@brlbil
Copy link
Contributor Author

brlbil commented Jan 26, 2024

/ci-aks

@brlbil
Copy link
Contributor Author

brlbil commented Jan 26, 2024

/ci-gke

@brlbil brlbil force-pushed the pr/brlbil/change-ci-regions branch from e1d3299 to 34ef603 Compare January 27, 2024 05:44
@brlbil
Copy link
Contributor Author

brlbil commented Jan 27, 2024

/ci-aks

@brlbil
Copy link
Contributor Author

brlbil commented Jan 27, 2024

/ci-gke

@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Feb 1, 2024
Merged via the queue into main with commit 2c29d8f Feb 1, 2024
205 checks passed
@aanm aanm deleted the pr/brlbil/change-ci-regions branch February 1, 2024 16:57
@nbusseneau nbusseneau mentioned this pull request Feb 8, 2024
Merged
6 tasks
@nbusseneau nbusseneau added backport-pending/1.13 The backport for Cilium 1.13.x for this PR is in progress. and removed needs-backport/1.13 This PR / issue needs backporting to the v1.13 branch labels Feb 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.13 in 1.13.12 Feb 8, 2024
@nbusseneau nbusseneau mentioned this pull request Feb 8, 2024
Merged
9 tasks
@nbusseneau nbusseneau added backport-pending/1.14 The backport for Cilium 1.14.x for this PR is in progress. and removed needs-backport/1.14 This PR / issue needs backporting to the v1.14 branch labels Feb 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.14 in 1.14.7 Feb 8, 2024
@nbusseneau nbusseneau mentioned this pull request Feb 8, 2024
Merged
12 tasks
@nbusseneau nbusseneau added backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. and removed needs-backport/1.15 This PR / issue needs backporting to the v1.15 branch labels Feb 8, 2024
@github-actions github-actions bot added backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. backport-done/1.14 The backport for Cilium 1.14.x for this PR is done. and removed backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. backport-pending/1.14 The backport for Cilium 1.14.x for this PR is in progress. labels Feb 9, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed this from Backport pending to v1.14 in 1.14.7 Feb 9, 2024
@github-actions github-actions bot added backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. and removed backport-pending/1.13 The backport for Cilium 1.13.x for this PR is in progress. labels Feb 9, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Backport done to v1.14 in 1.14.7 Feb 9, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed this from Backport pending to v1.13 in 1.13.12 Feb 9, 2024
This was referenced Feb 13, 2024
Merged
Merged
@michi-covalent michi-covalent moved this from Needs backport from main to Backport done to v1.15 in 1.15.1 Feb 14, 2024
@michi-covalent michi-covalent mentioned this pull request Feb 14, 2024
Merged
This was referenced Apr 3, 2024
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tklauser tklauser tklauser approved these changes

@aanm aanm aanm approved these changes

Assignees
No one assigned
Labels
area/CI Continuous Integration testing issue or flake area/CI-improvement Topic or proposal to improve the Continuous Integration workflow backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. backport-done/1.14 The backport for Cilium 1.14.x for this PR is done. backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/ci This PR makes changes to the CI.
Projects
No open projects
1.14.7
Backport done to v1.14
1.15.1
Backport done to v1.15
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CI: Conformance AWS (awscni) fails with "The maximum number of VPCs has been reached"
4 participants
@brlbil @tklauser @aanm @nbusseneau