Skip to content

Releases: kubernetes/kops

v1.25.0

19 Sep 12:04
@hakman hakman
v1.25.0
9b957a1
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.25.0

Significant changes

  • GCE cloud provider support has been promoted to stable.
  • Hetzner cloud provider support has been promoted to beta.
  • Karpenter support has been promoted to stable on Kubernetes versions 1.22, 1.23 and 1.24. Karpenter does not yet support Kubernetes above 1.25.
  • IAM roles on AWS used for ServiceAccounts are now tagged with the name and namespace of the ServiceAccount.
  • Cert Manager may now solve dns-01 challenges. See the cert manager documentation.
  • Add support to --cordon-node-before-terminating on the cluster autoscaler addon (CordonNodeBeforeTerminating)
  • EBS CSI driver can now be self-managed. See the addon docs.

Breaking changes

Cinder CSI snapthot controller changes

The CSI Cinder plugin for OpenStack will now only use the CSI snapshotter when the CSI snapshot controller is enabled in the cluster spec. This changes the default behavior where the CSI snaphotter container was always present, but spammed the log with error messages (see #13890). In case of manually deployed CRDs to make the snapshotter work it is now necessary to enable the snapshot controller.

Other breaking changes

  • Support for Kubernetes version 1.19 has been removed.

Deprecations

  • Support for Kubernetes version 1.20 is deprecated and will be removed in kOps 1.26.
  • Support for Kubernetes version 1.21 is deprecated and will be removed in kOps 1.27.

What's Changed

Read more

Contributors

olemarkus, justinsb, and 27 other contributors
Assets 24

v1.24.3

16 Sep 09:12
@olemarkus olemarkus
v1.24.3
dadb649
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.3

General release notes for kOps 1.24

What's Changed

Full Changelog: v1.24.2...v1.24.3

Contributors

olemarkus and hakman
Assets 24

v1.24.2

03 Sep 05:01
@hakman hakman
v1.24.2
422bfff
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.2

What's Changed

Full Changelog: v1.24.1...v1.24.2

Contributors

olemarkus, justinsb, and 2 other contributors
Assets 24

v1.23.4

03 Sep 04:35
@hakman hakman
v1.23.4
824158f
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.23.4

What's Changed

Full Changelog: v1.23.3...v1.23.4

Contributors

justinsb, hakman, and sterchelen
Assets 26

v1.25.0-beta.1

31 Aug 11:59
@hakman hakman
v1.25.0-beta.1
8b83ded
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.25.0-beta.1 Pre-release
Pre-release

What's Changed

New Contributors

Full Changelog: v1.25.0-alpha.2...v1.25.0-beta.1

Contributors

olemarkus, justinsb, and 9 other contributors
Assets 24

v1.25.0-alpha.2

29 Jul 19:33
@hakman hakman
v1.25.0-alpha.2
4dee7dd
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.25.0-alpha.2 Pre-release
Pre-release

What's Changed

New Contributors

Full Changelog: v1.25.0-alpha.1...v1.25.0-alpha.2

Contributors

olemarkus, justinsb, and 14 other contributors
Assets 24

v1.24.1

29 Jul 19:37
@hakman hakman
v1.24.1
0e1f09c
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.1

What's Changed

  • Automated cherry pick of #13901: Use Calico v3.23 for Kubernetes 1.22+ by @hakman in #13968
  • Automated cherry pick of #13965: Use control-plane node role for AWS IAM Authenticator by @rifelpet in #13967
  • Automated cherry pick of #13970: Skip deregistering the instance during rolling update for by @hakman in #13971
  • Automated cherry pick of #13979: Upgrade aws-iam-authenticator to v0.5.9 by @rifelpet in #13980
  • Automated cherry pick of #13982: Use only IPv4 for Hetzner servers by @hakman in #13984
  • Automated cherry pick of #13975: Add option to set etcd-manager backup interval by @hakman in #13983
  • Automated cherry pick of #13990: Update etcd-manager to v3.0.20220717 by @hakman in #13991
  • Automated cherry pick of #13994: Update Go to v1.18.4 by @hakman in #13996
  • Automated cherry pick of #13986: Add option to set number of replicas for pod-identity-webhook by @hakman in #13988
  • Automated cherry pick of #14005: Upgrade DO CSI driver to 4.2.0 by @hakman in #14006
  • Update k8s.io/client-go to match k8s.io/api by @hakman in #14003
  • Automated cherry pick of #14015: Switch to latest MacOS version for CI by @hakman in #14019
  • Automated cherry pick of #14024: Revert to using instance private DNS name to lookup hostname by @hakman in #14025
  • Automated cherry pick of #14018: Add server group management for Hetzner by @hakman in #14028
  • Update dependencies for kOps 1.24 by @hakman in #13989
  • Automated cherry pick of #13908: Update Calico to v3.23.2 #14009: Update Calico to v3.23.3 by @hakman in #14010
  • Automated cherry pick of #14038: Update etcd-manager to v3.0.20220727 by @hakman in #14039
  • Automated cherry pick of #14041: Check keyset existence before attempting to distrust by @hakman in #14042
  • Automated cherry pick of #14046: Fix SIGSEGV when deleting a Hetzner instance by @hakman in #14047
  • Automated cherry pick of #14053: Remove namespaces from cluster-scoped resources in CNI by @hakman in #14059
  • Automated cherry pick of #14034: Enable rolling updates for Hetzner
    #14057: Wait for load balancer to be ready for Hetzner
    #14058: Add multiple SSH keys support for Hetzner by @hakman in #14067
  • Automated cherry pick of #14054: Use cabundle for etcd CA files by @olemarkus in #14069
  • Release 1.24.1 by @hakman in #14071

Full Changelog: v1.24.0...v1.24.1

Contributors

olemarkus, rifelpet, and hakman
Assets 24

v1.23.3

29 Jul 14:25
@justinsb justinsb
v1.23.3
70da5cc
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.23.3

Release notes for kOps 1.23 series

Significant changes

  • If the Kubernetes version is 1.23 or later and the external AWS Cloud Controller Manager is
    being used, then Kubernetes Node resources will be named after their AWS instance ID instead of their domain name and
    managed subnets will be configured to launch instances with Resource Based Names.

  • Support for ShutdownGracePeriod and ShutdownGracePeriodCriticalPods. By default, kOps will set ShutdownGracePeriod to 30 seconds and ShutdownGracePeriodCriticalPods to 10 seconds if the Kubernetes version is above 1.21.

  • By enabling the pod identity webhook, you no longer need to modify your Pod specs to assume IAM roles.

Breaking changes

  • Support for Kubernetes version 1.17 has been removed.

  • Support for the Lyft CNI has been removed.

  • The Weave CNI is not supported for Kubernetes 1.23 or later.

  • Support for CentOS 7 has been removed.

  • Support for CentOS 8 has been removed (replaced by Rocky Linux 8).

  • Support for Debian 9 has been removed.

  • Support for RHEL 7 is has been removed.

  • Support for Ubuntu 16.04 (Xenial) has been removed.

  • Cilium now has disable-cnp-status-updates: true by default. Set this to false if you rely on the CiliumNetworkPolicy status fields.

Required actions

Deprecations

  • Support for Kubernetes version 1.18 is deprecated and will be removed in kOps 1.24.

  • Support for Kubernetes version 1.19 is deprecated and will be removed in kOps 1.25.

  • All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.

  • The node-role.kubernetes.io/master and kubernetes.io/role labels are deprecated and might be removed from control plane nodes in future versions of kOps.

  • Due to lack of maintainers, the Aliyun/Alibaba Cloud support has been deprecated and will be removed in kOps 1.24.

  • Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

Other changes of note

  • The kops create cluster command has a new --discovery-store flag for specifying a public store for the OIDC-compatible discovery documents.
    If this flag is used in AWS, it will enable IRSA.

  • If externalDns.provider is external-dns, then externalDns.watchIngress will now default to true.

  • This release introduces a v1alpha3 API version. This API version is a work in progress and is likely to be replaced in kOps 1.24.
    It is recommended to keep using the v1alpha2 API version.

  • IPv6 pod subnets is in a working state using public IPv6 addresses for the Pod network. This works with both Cilium and Calico. IPv6 is still behind a feature flag until service controllers and addons implement support for IPv6. See the IPv6 documentation.

  • The kops rolling-update cluster command has a new --drain-timeout flag for specifying the maximum amount of time to wait when attempting to drain a node. Previously, rolling-updates would attempt to drain a node for an indefinite amount of time. If --drain-timeout is not specified, a default of 15 minutes is applied.

  • Fix inconsistent output of kops get clusters -ojson. This will now always return a list (irrespective of a single or multiple clusters) to keep the format consistent. However, note that kops get cluster dev.example.com -ojson will continue to work as previously, and will return a single object.

  • Digital Ocean kops now has vpc support. You can specify a network-cidr range while creating the kops cluster. kops resources will be created in the new vpc range. Also supports shared vpc; you can specify the vpc uuid while creating kops cluster.

1.23.2 to 1.23.3

Assets 26

v1.22.6

29 Jul 14:23
@justinsb justinsb
v1.22.6
3a6f211
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.22.6

Release notes for kOps 1.22 series

Significant changes

Instance metadata service version 2

On AWS, kOps will enable Instance Metadata Service Version 2 and require tokens on new clusters with Kubernetes 1.22. In addition, the following max hop limits will be set by default:

  • worker and API server Nodes, and bastions, will have a limit of 1 hop.
  • control plane nodes will have a limit of 3 hops to accommodate for controller Pods without host networking that need to assume roles.

This will increase security by default, but may break some types of workloads. In order to revert to old behavior, add the following to the InstanceGroup:

spec:
  instanceMetadata:
    httpTokens: optional

External ServiceAccountPermissions

Many of kOps addons can now make direct use of external permissions.
This can be enabled by adding the following to the Cluster spec:

spec:
  iam:
    useServiceAccountExternalPermissions: true

Currently this is only available using the AWS cloud provider.

Managed nvidia instances

kOps can now provision instances with nvidia GPUs and configure it for container workloads without the need of hooks and operators. See GPU support

Breacking change in NodeLocalDNS

Since 1.22.0 Cluster spec.kubeDNS.nodeLocalDNS.forwardToKubeDNS default behaviour changes from true to false.

Other significant changes

  • New clusters on AWS will no longer provision an SSH public key by default. To provision
    an SSH public key on a new cluster, use the --ssh-public-key flag to kops create cluster.

  • The kOps Terraform support now renders managed files through the Terraform configuration instead
    of writing them to S3 directly. This defers changes to these files until the time of terraform apply.
    This feature may be temporarily disabled by turning off the TerraformManagedFiles feature flag
    using export KOPS_FEATURE_FLAGS="-TerraformManagedFiles".

  • kOps now implements graceful rotation of its Certificate Authorities and the service
    account signing key. See the documentation on How to rotate all secrets / credentials

  • New clusters running Kubernetes 1.22 will have AWS EBS CSI driver enabled by default.

  • kOps now supports Debian 11 (Bullseye).

  • kOps can now use external-dns as a drop-in replacement for dns-controller.

Breaking changes

Control plane pods no longer mount /srv/kubernetes

For security reasons, /srv/kubernetes is no longer mounted in the kube-apiserver and kube-controller-manager Pods. This also means the files in the default file assets path will be unavailable. If you have file assets or other files needed by kube-apiserver, you must put these into /srv/kubernetes/kube-apiserver/ or /srv/kubernetes/kube-controller-manager, respectively.

For file assets, it means adding an explicit path as shown below:

  fileAssets:
  - name: audit-policy-config
    path: /srv/kubernetes/kube-apiserver/audit-policy-config.yaml # make sure you add the path
    roles:
    - Master
    content: |
      apiVersion: audit.k8s.io/v1
      kind: Policy
      rules:
      - level: Metadata

Other breaking changes

  • Support for Kubernetes versions 1.15 and 1.16 has been removed.

  • The legacy addons from https://github.com/kubernetes/kops/tree/master/addons have been deprecated and will not be available in Kubernetes 1.23+. Use managed addons instead.

  • The legacy location for downloads s3://https://kubeupv2.s3.amazonaws.com/kops/ has been deprecated and will not be used for new releases. The new canonical downloads location is https://artifacts.k8s.io/binaries/kops/.

  • The assets phase of kops update cluster has been removed. It is replaced by the new kops get assets --copy command.

  • Support for importing and converting kubeup clusters has been removed.

  • Support for Cilium and RHEL 8 has been removed. Cilium users will need to migrate to a distribution with a newer Linux kernel.

Required actions

  • Amazon Linux 2 users are encouraged to use the AMIs based on the 5.10 Linux kernel. See the documentation for more information.

  • Terraform support now requires Terraform >=0.15.0.
    Users on older versions must follow Terraform's recommended upgrade path of applying one minor version at a time prior to running kops update cluster --target terraform.

  • The kOps Terraform support now renders managed files through the Terraform configuration instead
    of writing them to S3 directly. If, after upgrading kOps and applying a new Terraform plan,
    you subsequently downgrade to an earlier version of kOps, the generated plan will delete these
    files, breaking the cluster. Prior to applying the plan, you will need to orphan all the
    aws_s3_bucket_object objects the plan wants to destroy. Use terraform state rm on each of them.
    Then re-run terraform plan until there are no such objects in the plan.

    If you applied the plan without first orphaning all of these objects, fix the cluster by re-running
    kops update cluster --target terraform.

  • Terraform users of clusters with names beginning with digits will need to move resources prior to upgrading to kOps 1.22. Some of the following commands will need to be run depending on the particular cluster configuration. Confirm the Terraform plan doesn't destroy any of these resources before running terraform apply.

    # View the existing terraform resource names for the exact value to use
HYPHENATED_CLUSTER_NAME=123-cluster-example-com
terraform state mv "aws_iam_openid_connect_provider.${HYPHENATED_CLUSTER_NAME}" "aws_iam_openid_connect_provider.prefix_${HYPHENATED_CLUSTER_NAME}"
terraform state mv "aws_internet_gateway.${HYPHENATED_CLUSTER_NAME}" "aws_internet_gateway.prefix_${HYPHENATED_CLUSTER_NAME}"
terraform state mv "aws_route_table.${HYPHENATED_CLUSTER_NAME}" "aws_route_table.prefix_${HYPHENATED_CLUSTER_NAME}"
terraform state mv "aws_vpc.${HYPHENATED_CLUSTER_NAME}" "aws_vpc.prefix_${HYPHENATED_CLUSTER_NAME}"
terraform state mv "aws_vpc_dhcp_options.${HYPHENATED_CLUSTER_NAME}" "aws_vpc_dhcp_options.prefix_${HYPHENATED_CLUSTER_NAME}"
terraform state mv "aws_vpc_dhcp_options_association.${HYPHENATED_CLUSTER_NAME}" "aws_vpc_dhcp_options_association.prefix_${HYPHENATED_CLUSTER_NAME}"

Deprecations

  • Support for Kubernetes version 1.17 is deprecated and will be removed in kOps 1.23.

  • Support for Kubernetes version 1.18 is deprecated and will be removed in kOps 1.24.

  • Support for the Lyft CNI is deprecated and will be removed in kOps 1.23.

  • Support for CentOS 7 is deprecated and will be removed in future versions of kOps.

  • Support for CentOS 8 is deprecated and will be removed in future versions of kOps.

  • Support for Debian 9 (Stretch) is deprecated and will be removed in future versions of kOps.

  • Support for RHEL 7 is deprecated and will be removed in future versions of kOps.

  • Support for Ubuntu 18.04 (Bionic) is deprecated and will be removed in future versions of kOps.

  • All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.

  • The node-role.kubernetes.io/master and kubernetes.io/role labels are deprecated and might be removed from control plane nodes in kOps 1.23.

  • The TerraformJSON feature flag is deprecated and will be removed in kOps 1.23. Only native HCL2 Terraform output will be supported.

  • Due to lack of maintainers, the Aliyun/Alibaba Cloud support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this cloud provider.

  • Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

Other changes of note

  • Support for shell completion has been substantially improved. kOps has added support for shell completion in fish and PowerShell.

  • It is no longer necessary to set AWS_SDK_LOAD_CONFIG=1 in the environment when using AWS assumed roles with the kops CLI.

  • There is a new command kops get assets for listing image and file assets used by a cluster.
    It also includes a --copy flag to copy the assets to local repositories.
    See the documentation on Using local asset repositories for more information.

  • kOps now provisions TLS server certificates signed by the Kubernetes general CA to kube-controller-manager and kube-scheduler.
    The previous behavior of using self-signed certs may be restored by setting kubeControllerManager.tlsCertFile and/or
    kubeScheduler.tlsCertFile to "" in the cluster spec.

  • Cilium now supports the wireguard protocol for transparent encryption.

1.22.5 to 1.22.6

  • Add hashes for containerd and Docker in order to fix CVE-2022-23648 @drequena #13606
  • Avoid "/etc/resolv.conf" file loopback for Flatcar Container Linux distribution @seh #13617
  • Update etcd-manager to v3...
Read more
Assets 26

v1.24.0

08 Jul 08:33
@hakman hakman
v1.24.0
59360d7
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.0

Release Notes

https://kops.sigs.k8s.io/releases/1.24-notes/

What's Changed

Read more

Contributors

glebiller, olemarkus, and 50 other contributors
Assets 24