etcd-io Infra and CI Migration

[upodroid]

etcd is now a subproject of Kubernetes and etcd maintainers are looking to adopt the CI system and Infra management approach we use for Kubernetes.

GitHub

Beta Give feedback

1. Add etcd-io org org#4498
   approved area/github-management cncf-cla: yes lgtm sig/etcd size/L +6
   

Infra

Beta Give feedback

1. Manage the DNS for etcd.io using GitOps #6600
   +0
2. Enumerate all the GCP projects used by etcd and discuss what to do with them. Some GCP resources are mentioned in the etcd releases notes. https://github.com/etcd-io/etcd/releases/tag/v3.5.10
3. Enumerate all the *.etcd.io services and migrate them to the community cluster. https://github.com/kubernetes/k8s.io/blob/main/running-in-community-clusters.md
4. Deploy arm64 nodepools on k8s-infra-prow-build GKE cluster. etcd has a requirement to run e2e tests on arm64 hardware, Switch over to managed Arm64 hosts etcd-io/etcd#16801
5. Explore serving etcd images at registry.k8s.io and resolve the mismatch between https://explore.ggcr.dev/?repo=registry.k8s.io%2Fetcd and https://explore.ggcr.dev/?repo=gcr.io%2Fetcd-development%2Fetcd
6. Verify if etcd.io domain is being used for GSuite. If not, attach it to the kubernetes.io gsuite and create the relevant mailing lists for it.

Testing

Beta Give feedback

1. Bootstrap etcd jobs on prow test-infra#31218
   approved area/config area/jobs area/testgrid cncf-cla: yes lgtm sig/testing size/L +8
   
2. add e2e etcd presubmit test-infra#31257
   approved area/config area/jobs cncf-cla: yes lgtm sig/etcd sig/testing size/M +8
   
3. Enable additional prow plugins, particularly tide, lgtm and approve plugins.
4. Create arm64 version of the kubekins-e2e and bootstrap image test-infra#31273
   kind/feature sig/testing +2
5. Create kubekins v2 image test-infra#31421
   approved area/images area/release-eng cncf-cla: yes lgtm sig/release sig/testing size/L +8
   
6. Fix the main etcd e2e job https://testgrid.k8s.io/sig-etcd-periodics#ci-etcd-e2e-amd64
7. Create a prowjob for all the github actions at https://github.com/etcd-io/etcd/tree/main/.github/workflows

If I missed something, feel free to comment on the issue and I'll update the tracker.

/cc @jmhbnz @serathius @wenjiaswe @mrbobbytables @ahrtr @ameukam @BenTheElder

/sig etcd
/sig testing
/priority important-soon
/kind feature

[serathius]

Have we stopped running presubmits? I stopped seeing them in Github PRs and https://testgrid.k8s.io/sig-etcd-presubmits seems empty. Nothing in https://prow.k8s.io/?repo=etcd-io%2Fetcd too

[upodroid]

There is a thread in #testing-ops on Slack to investigate this issue
https://kubernetes.slack.com/archives/C7J9RP96G/p1700688510160169

[serathius]

Interesting flake in unit test:

=== FAIL: storage/schema TestMigrate/Upgrading_3.6_to_v3.7_is_not_supported (0.01s)
    logger.go:130: 2023-11-23T15:59:05.436Z	WARN	failed to preallocate an initial WAL file	{"path": "/tmp/TestMigrateUpgrading_3.6_to_v3.7_is_not_supported2935238209/002/etcd_wal_test401442[383](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/etcd-io_etcd/17008/pull-etcd-unit-test/1727717754938593280#1:build-log.txt%3A383)7/wal.tmp/0000000000000000-0000000000000000.wal", "segment-bytes": 64000000, "error": "no space left on device"}
    schema_test.go:207: Failed to create WAL: no space left on device
    --- FAIL: TestMigrate/Upgrading_3.6_to_v3.7_is_not_supported (0.01s)

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/etcd-io_etcd/17008/pull-etcd-unit-test/1727717754938593280

[upodroid]

Does that unit test expect the pod to have an ephemeral volume of a specific size?

[serathius]

I think problem might stem from etcd WAL tests. I don't think that unit tests mock storage, just write to t.TempDir() (should be /tmp/ by default). WAL creation pre-allocates 64MB, so if there are couple of such tests running without cleanup we could be allocating couple of hundreds of megabytes.

[wenjiaswe]

cc @siyuanfoundation

[BenTheElder]

we could mount an emptyDir (disk or memory) to /tmp if etcd tests are writing to it heavily.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[upodroid]

/lifecycle frozen

[serathius]

Note for robustness tests, there is one functionally to github actions that was not migrated. Uploading of test report artifacts.

[cblecker]

etcd DNS was migrated in #6600.

Today I have migrated the etcd netlify site to the Kubernetes account, and requested the CNCF close the etcd netlify account in CNCFSD-2245

[ArkaSaha30]

Hello 👋
I am willing to take up some of the workflow issues for migration to Prowjobs