After upgrade of our EKS cluster, Spinnaker pipeline config was lost and Rosco (Packer) spun up hundreds EC2 instances

[jbimela]

Issue Summary:

Our Spinnaker v1.28.1 application runs on EKS and managed by Halyard. Redis is used for caching. After upgrading EKS from v1.26 to v1.27 ( See https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions-standard.html), all our pipeline configuration and cluster information were lost. Packer builder on the other hand started baking images of all our applications resulting in hundreds of EC2 instances being created.

Cloud Provider(s):

AWS

Environment:

Spinnaker 1.28.1, Kubernetes 1.27

Feature Area:

UI pipeline, Rosco

Description:

In the previous EKS upgrades that we have done the past, all application pipeline history and cluster configuration have been preserved. Image bakery has always been triggered leading to the creation of few EC2 instances. However, the last EKS upgrade we performed let to the loss of cluster and pipeline history, the creation of hundreds of EC2 instances and jobs were triggered.

---

[spinnakerbot]

This issue hasn't been updated in 45 days, so we are tagging it as 'stale'. If you want to remove this label, comment:

@spinnakerbot remove-label stale

[spinnakerbot]

This issue is tagged as 'stale' and hasn't been updated in 45 days, so we are tagging it as 'to-be-closed'. It will be closed in 45 days unless updates are made. If you want to remove this label, comment:

@spinnakerbot remove-label to-be-closed