eks-prow-build-cluster: Decommission the canary cluster

[xmudrii]

We have something called the EKS Prow build canary cluster. Originally, it has been a test cluster for all sort of changes, but we are running into issues lately:

• It costs money that we're just giving away to AWS, we don't get that much value from the cluster
• We're very limited in terms what we can test there because that's not actually a Prow cluster, we can't run Prow jobs that easily there
• It's a sort of a maintenance burden and we don't have such a cluster on GCP
• And to the main point, this is making GitOps very hard. We now want to manage more manifests with Flux and we're facing issues that we have places where we need to put the cluster name and the account ID in manifests, which is not really nice to get working with Flux

We discussed this on Slack and decided to decommission the canary cluster. There are several tasks to take care of:

Tasks

Beta Give feedback

1. Destroy the cluster and the infrastructure (terraform destroy)
2. Decommission the AWS account
3. Remove canary cluster from Terraform and Makefile
4. Update docs to remove references to the canary cluster

/assign @xmudrii @koksay
/sig k8s-infra
/area infra infra/aws
/milestone v1.30
/priority important-soon

[BenTheElder]

1.30 is out :-)

[xmudrii]

/milestone v1.31