Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Openshift E2E test compatibility #6391

Merged

Conversation

wallrj
Copy link
Member

@wallrj wallrj commented Oct 5, 2023

Following up on #6387 I've made some further changes to allow me to run the compiled e2e.test binary against an OpenShift cluster.

Testing

Checked that make e2e configures vault to use the local image

richard@LAPTOP-HJEQ9V9G:~/projects/cert-manager/cert-manager$ make e2e GINKGO_FOCUS="CA\ Issuer"
cd _bin/tools/ && ln -f -s ../downloaded/tools/ginkgo@v2.12.0_linux_amd64 ginkgo
make/e2e.sh
ginkgo --tags=e2e_test --procs=1 --output-dir=/home/richard/projects/cert-manager/cert-manager/_bin/artifacts --junit-report=junit__01.xml --flake-attempts=1 --timeout=1h -v --randomize-all --trace --poll-progress-after=60s ./test/e2e/ -- --repo-root=/home/richard/projects/cert-manager/cert-manager --report-dir=/home/richard/projects/cert-manager/cert-manager/_bin/artifacts --acme-dns-server=10.0.0.16 --acme-ingress-ip=10.0.0.15 --acme-gateway-ip=10.0.0.14 --ingress-controller-domain=ingress-nginx.http01.example.com --gateway-domain=gateway.http01.example.com --feature-gates=AdditionalCertificateOutputFormats=true,ExperimentalCertificateSigningRequestControllers=true,ExperimentalGatewayAPISupport=true,LiteralCertificateSubject=true "--ginkgo.focus=CA\ Issuer" --ginkgo.v --test.v
=== RUN   TestE2E
Running Suite: cert-manager e2e suite - /home/richard/projects/cert-manager/cert-manager/test/e2e
=================================================================================================
Random Seed: 1696510643 - will randomize all specs

Will run 45 of 777 specs
------------------------------
[SynchronizedBeforeSuite]
/home/richard/projects/cert-manager/cert-manager/test/e2e/e2e.go:40
"hashicorp" already exists with the same configuration, skipping
Release "chart-vault-vault" does not exist. Installing it now.
NAME: chart-vault-vault
LAST DEPLOYED: Thu Oct  5 13:57:30 2023
NAMESPACE: e2e-vault
STATUS: deployed
REVISION: 1
NOTES:
Thank you for installing HashiCorp Vault!

Now that you have deployed Vault, you should look over the docs on using
Vault with Kubernetes available here:

https://www.vaultproject.io/docs/


Your release is named chart-vault-vault. To learn more about the release, try:

  $ helm status chart-vault-vault
  $ helm get manifest chart-vault-vault
[SynchronizedBeforeSuite] PASSED [11.957 seconds]

...

------------------------------
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[SynchronizedAfterSuite]
/home/richard/projects/cert-manager/cert-manager/test/e2e/e2e.go:85
  STEP: Retrieving logs for global addons @ 10/05/23 13:58:42.395
  STEP: Cleaning up the provisioned globals @ 10/05/23 13:58:42.45
proxy logs:
[SynchronizedAfterSuite] PASSED [0.455 seconds]
------------------------------
[ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report
autogenerated by Ginkgo
[ReportAfterSuite] PASSED [0.029 seconds]
------------------------------

Ran 43 of 777 Specs in 73.513 seconds
SUCCESS! -- 43 Passed | 0 Failed | 0 Pending | 734 Skipped
--- PASS: TestE2E (73.55s)
PASS

Ginkgo ran 1 suite in 1m19.043566056s
Test Suite Passed
richard@LAPTOP-HJEQ9V9G:~$ kubectl -n e2e-vault describe pod chart-vault-vault-0
...
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  13s   default-scheduler  Successfully assigned e2e-vault/chart-vault-vault-0 to kind-control-plane
  Normal  Pulled     13s   kubelet            Container image "local/vault:local" already present on machine
  Normal  Created    13s   kubelet            Created container vault
  Normal  Started    13s   kubelet            Started container vault

Tested against an OpenShift 4.13 cluster with cert-manager installed via OperatorHub

[crc@crc-4-13 ~]$ OPENSHIFT=true ./e2e --repo-root=/dev/null --ginkgo.focus="Vault\ Issuer" --ginkgo.skip="Gateway"
Running Suite: cert-manager e2e suite - /home/crc
=================================================
Random Seed: 1696508729

Will run 25 of 777 specs
"hashicorp" already exists with the same configuration, skipping
Release "chart-vault-vault" does not exist. Installing it now.
NAME: chart-vault-vault
LAST DEPLOYED: Thu Oct  5 12:25:31 2023
NAMESPACE: e2e-vault
STATUS: deployed
REVISION: 1
NOTES:
Thank you for installing HashiCorp Vault!

Now that you have deployed Vault, you should look over the docs on using
Vault with Kubernetes available here:

https://www.vaultproject.io/docs/


Your release is named chart-vault-vault. To learn more about the release, try:

  $ helm status chart-vault-vault
  $ helm get manifest chart-vault-vault
••••[controller-runtime] log.SetLogger(...) was never called; logs will not be displayed.
Detected at:
        >  goroutine 152 [running]:
        >  runtime/debug.Stack()
        >       runtime/debug/stack.go:24 +0x5e
        >  sigs.k8s.io/controller-runtime/pkg/log.eventuallyFulfillRoot()
        >       sigs.k8s.io/controller-runtime@v0.16.2/pkg/log/log.go:60 +0xcd
        >  sigs.k8s.io/controller-runtime/pkg/log.(*delegatingLogSink).WithName(0xc000149e40, {0x1da69f3, 0x14})
        >       sigs.k8s.io/controller-runtime@v0.16.2/pkg/log/deleg.go:147 +0x45
        >  github.com/go-logr/logr.Logger.WithName({{0x2070db0, 0xc000149e40}, 0x0}, {0x1da69f3?, 0xc000a79d98?})
        >       github.com/go-logr/logr@v1.2.4/logr.go:336 +0x3d
        >  sigs.k8s.io/controller-runtime/pkg/client.newClient(0x0?, {0x0, 0xc000345110, {0x0, 0x0}, 0x0, {0x0, 0x0}, 0x0})
        >       sigs.k8s.io/controller-runtime@v0.16.2/pkg/client/client.go:122 +0xec
        >  sigs.k8s.io/controller-runtime/pkg/client.New(0x1dc1053?, {0x0, 0xc000345110, {0x0, 0x0}, 0x0, {0x0, 0x0}, 0x0})
        >       sigs.k8s.io/controller-runtime@v0.16.2/pkg/client/client.go:103 +0x7d
        >  github.com/cert-manager/cert-manager/e2e-tests/framework.(*Framework).BeforeEach(0xc00064de00)
        >       github.com/cert-manager/cert-manager/e2e-tests/framework/framework.go:145 +0x33c
        >  github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x0, 0x0})
        >       github.com/onsi/ginkgo/v2@v2.12.0/internal/node.go:463 +0x13
        >  github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
        >       github.com/onsi/ginkgo/v2@v2.12.0/internal/suite.go:865 +0x8d
        >  created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 11
        >       github.com/onsi/ginkgo/v2@v2.12.0/internal/suite.go:852 +0xd7b
•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•••••SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•••••••••
------------------------------
• [FAILED] [61.016 seconds]
[cert-manager] Vault Issuer [It] should be ready with a valid serviceAccountRef
github.com/cert-manager/cert-manager/e2e-tests/suite/issuers/vault/issuer.go:346

  Timeline >>
  STEP: Creating a kubernetes client @ 10/05/23 12:27:00.888
  STEP: Creating an API extensions client @ 10/05/23 12:27:00.89
  STEP: Creating a cert manager client @ 10/05/23 12:27:00.89
  STEP: Creating a controller-runtime client @ 10/05/23 12:27:00.89
  STEP: Creating a gateway-api client @ 10/05/23 12:27:00.891
  STEP: Building a namespace api object @ 10/05/23 12:27:00.891
  STEP: Using the namespace e2e-tests-create-vault-issuer-z524j @ 10/05/23 12:27:00.909
  STEP: Building a ResourceQuota api object @ 10/05/23 12:27:00.909
  STEP: Configuring the Vault server @ 10/05/23 12:27:00.952
  STEP: creating a service account for Vault authentication @ 10/05/23 12:27:01.489
  STEP: Creating the Role and RoleBinding to let cert-manager use TokenRequest for the ServiceAccount @ 10/05/23 12:27:01.566
  STEP: Creating an Issuer @ 10/05/23 12:27:01.662
  STEP: Waiting for Issuer to become Ready @ 10/05/23 12:27:01.721
  Oct  5 12:27:01.721: INFO: Waiting for issuer test-vault-issuer condition v1.IssuerCondition{Type:"Ready", Status:"True", LastTransitionTime:<nil>, Reason:"", Message:"", ObservedGeneration:0}
  Oct  5 12:27:06.870: INFO: Waiting for issuer test-vault-issuer condition v1.IssuerCondition{Type:"Ready", Status:"True", LastTransitionTime:<nil>, Reason:"", Message:"", ObservedGeneration:0}
  Oct  5 12:27:13.368: INFO: Waiting for issuer test-vault-issuer condition v1.IssuerCondition{Type:"Ready", Status:"True", LastTransitionTime:<nil>, Reason:"", Message:"", ObservedGeneration:0}
  Oct  5 12:27:20.868: INFO: Waiting for issuer test-vault-issuer condition v1.IssuerCondition{Type:"Ready", Status:"True", LastTransitionTime:<nil>, Reason:"", Message:"", ObservedGeneration:0}
  Oct  5 12:27:29.868: INFO: Waiting for issuer test-vault-issuer condition v1.IssuerCondition{Type:"Ready", Status:"True", LastTransitionTime:<nil>, Reason:"", Message:"", ObservedGeneration:0}
  Oct  5 12:27:40.368: INFO: Waiting for issuer test-vault-issuer condition v1.IssuerCondition{Type:"Ready", Status:"True", LastTransitionTime:<nil>, Reason:"", Message:"", ObservedGeneration:0}
  Oct  5 12:27:52.868: INFO: Waiting for issuer test-vault-issuer condition v1.IssuerCondition{Type:"Ready", Status:"True", LastTransitionTime:<nil>, Reason:"", Message:"", ObservedGeneration:0}
  Oct  5 12:28:01.731: INFO: Waiting for issuer test-vault-issuer condition v1.IssuerCondition{Type:"Ready", Status:"True", LastTransitionTime:<nil>, Reason:"", Message:"", ObservedGeneration:0} (took 1m0s)
  [FAILED] in [It] - github.com/cert-manager/cert-manager/e2e-tests/suite/issuers/vault/issuer.go:370 @ 10/05/23 12:28:01.786
  STEP: Cleaning up AppRole @ 10/05/23 12:28:01.786
  STEP: Cleaning up Kubernetes @ 10/05/23 12:28:01.856
  STEP: Cleaning up Vault @ 10/05/23 12:28:01.862
  STEP: Deleting test namespace @ 10/05/23 12:28:01.88
  << Timeline

  [FAILED] Unexpected error:
      <*errors.errorString | 0xc000db9550>:
      context deadline exceeded: Last Status: 'False' Reason: 'VaultError', Message: 'Failed to initialize Vault client: while requesting a Vault token using the Kubernetes auth: while requesting a token for the service account e2e-tests-create-vault-issuer-z524j/vault-serviceaccount: serviceaccounts "vault-serviceaccount" is forbidden: User "system:serviceaccount:openshift-operators:cert-manager" cannot create resource "serviceaccounts/token" in API group "" in the namespace "e2e-tests-create-vault-issuer-z524j"'
      {
          s: "context deadline exceeded: Last Status: 'False' Reason: 'VaultError', Message: 'Failed to initialize Vault client: while requesting a Vault token using the Kubernetes auth: while requesting a token for the service account e2e-tests-create-vault-issuer-z524j/vault-serviceaccount: serviceaccounts \"vault-serviceaccount\" is forbidden: User \"system:serviceaccount:openshift-operators:cert-manager\" cannot create resource \"serviceaccounts/token\" in API group \"\" in the namespace \"e2e-tests-create-vault-issuer-z524j\"'",
      }
  occurred
  In [It] at: github.com/cert-manager/cert-manager/e2e-tests/suite/issuers/vault/issuer.go:370 @ 10/05/23 12:28:01.786
------------------------------
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•••••proxy logs:


Summarizing 1 Failure:
  [FAIL] [cert-manager] Vault Issuer [It] should be ready with a valid serviceAccountRef
  github.com/cert-manager/cert-manager/e2e-tests/suite/issuers/vault/issuer.go:370

Ran 25 of 777 Specs in 169.298 seconds
FAIL! -- 24 Passed | 1 Failed | 0 Pending | 752 Skipped
--- FAIL: TestE2E (169.32s)
FAIL
The end-to-end tests can now test the cert-manager Vault Issuer on an OpenShift cluster.

/kind cleanup

@jetstack-bot jetstack-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. area/testing Issues relating to testing labels Oct 5, 2023
Key: "global.openshift",
Value: "true",
},
}...,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this setting the Vault chart fails to install:

Release "chart-vault-vault" does not exist. Installing it now.
W1005 13:06:54.265120    6895 warnings.go:70] would violate PodSecurity "restricted:v1.24": unrestricted capabilities (container "vault" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or container "vault" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
NAME: chart-vault-vault

Copy link
Member

@inteon inteon Oct 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wallrj could we instead set the capabilities.drop = "ALL" and seccompProfile RuntimeDefault regardless of whether we are running on openshift?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could, but the helm chart makes other changes to the manifests contingent on the global.openshift value.

https://github.com/hashicorp/vault-helm/blob/e2990d291319b2097c9366918078263343173755/templates/_helpers.tpl#L545-L579

The advantage of my way is that we're using the chart as the authors intended, which makes the test more realistic and ensures that we will get any future OpenShift improvements that are made in the chart and we will be using the chart recommended defaults on our E2E Kind cluster.

The advantage of your suggestion is that we wouldn't need the E2E_OPENSHIFT flag.

I will leave it like this for now.

test/e2e/framework/addon/vault/vault.go Show resolved Hide resolved
## settings appropriate for OpenShift.
##
## @category Development
E2E_OPENSHIFT ?= false
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wallrj wallrj requested a review from inteon October 5, 2023 13:14
LOCALIMAGE_vaultretagged := local/vault:local
# Export this variable so that the image reference can be used by the Vault
# addon in the E2E tests
export LOCALIMAGE_vaultretagged := local/vault:local
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we maybe introduce a new variable that equals LOCALIMAGE_vaultretagged but is exported (eg. export E2E_VAULT_IMAGE)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its quite confusing that this one variable is exported and the other two LOCALIMAGE variables are not.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this breaks the flow where you would "make -j e2e-setup" and go debug afterwards (without using make in 2nd step).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its quite confusing that this one variable is exported and the other two LOCALIMAGE variables are not.

Yeah, I agree. I'll try and change it.

Also, this breaks the flow where you would "make -j e2e-setup" and go debug afterwards (without using make in 2nd step)

I don't think this change breaks that flow.
That flow was broken by my previous PR #6387

In that flow, you could still use go test ./test/e2e/... in the second step.
But it would install the Vault helm chart using the docker.io images rather than those that had been loaded into the kind cluster by `make e2e-setup.
Which isn't ideal, but it would only break if you're trying to run the tests offline.

…ary from my OLM Makefile

Signed-off-by: Richard Wall <richard.wall@venafi.com>
Signed-off-by: Richard Wall <richard.wall@venafi.com>
Signed-off-by: Richard Wall <richard.wall@venafi.com>
@wallrj wallrj force-pushed the openshift-e2e-compatibility branch from 1a9460c to a02c36f Compare October 5, 2023 14:58
@wallrj wallrj requested a review from inteon October 5, 2023 15:01
make/e2e-setup.mk Outdated Show resolved Hide resolved
Co-authored-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
Signed-off-by: Richard Wall <wallrj@users.noreply.github.com>
@inteon
Copy link
Member

inteon commented Oct 6, 2023

/approve
/lgtm
Thanks for making our tests work on openshift!

@jetstack-bot jetstack-bot added the lgtm Indicates that a PR is ready to be merged. label Oct 6, 2023
@jetstack-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: inteon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jetstack-bot jetstack-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 6, 2023
@jetstack-bot jetstack-bot merged commit d8035e6 into cert-manager:master Oct 6, 2023
6 checks passed
@wallrj wallrj deleted the openshift-e2e-compatibility branch October 6, 2023 14:37
nrdufour added a commit to nrdufour/home-ops that referenced this pull request Feb 4, 2024
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [cert-manager](https://github.com/cert-manager/cert-manager) | minor | `v1.13.3` -> `v1.14.1` |

---

### Release Notes

<details>
<summary>cert-manager/cert-manager (cert-manager)</summary>

### [`v1.14.1`](https://github.com/cert-manager/cert-manager/releases/tag/v1.14.1)

[Compare Source](cert-manager/cert-manager@v1.14.0...v1.14.1)

cert-manager is the easiest way to automatically manage certificates in Kubernetes and OpenShift clusters.

cert-manager 1.14 brings a variety of features, security improvements and bug fixes, including: support for creating X.509 certificates with "Other Name" fields, and support for creating CA certificates with "Name Constraints" and "Authority Information Accessors" extensions.

> 📢 cert-manager `v1.14.1` fixes bugs found *during* the release of `v1.14.0`.
>
> When upgrading to cert-manager release 1.14, please skip `v1.14.0` and install this patch version instead.

##### Documentation

-   [Release notes](https://cert-manager.io/docs/releases/release-notes/release-notes-1.14)
-   [Upgrade notes](https://cert-manager.io/docs/releases/upgrading/upgrading-1.13-1.14)
-   [Installation instructions](https://cert-manager.io/docs/installation/)

##### Changes since `v1.14.0`

##### Bug or Regression

-   Fix broken cainjector image value in Helm chart ([#&#8203;6693](cert-manager/cert-manager#6693), [@&#8203;SgtCoDFish](https://github.com/SgtCoDFish))
-   Fix bug in cmctl namespace detection which prevented it being used as a startupapicheck image in namespaces other than cert-manager. ([#&#8203;6706](cert-manager/cert-manager#6706), [@&#8203;inteon](https://github.com/inteon))
-   Fix bug in cmctl which caused `cmctl experimental install` to panic. ([#&#8203;6706](cert-manager/cert-manager#6706), [@&#8203;inteon](https://github.com/inteon))

### [`v1.14.0`](https://github.com/cert-manager/cert-manager/releases/tag/v1.14.0)

[Compare Source](cert-manager/cert-manager@v1.13.3...v1.14.0)

cert-manager is the easiest way to automatically manage certificates in Kubernetes and OpenShift clusters.

cert-manager 1.14 brings a variety of features, security improvements and bug fixes, including: support for creating X.509 certificates with "Other Name" fields, and support for creating CA certificates with "Name Constraints" and "Authority Information Accessors" extensions.

> ⚠️ This version has known issues. Please install `v1.14.1` instead.
>
> During the release of `v1.14.0`, the Helm chart was found to use the wrong OCI image for the `cainjector` Deployment,
> which caused the Helm installation and the static manifest based installation to fail.
> Upon discovery of this bug, the release of `v1.14.0` was paused before the Helm chart or GitHub release were published;
> but the Git tag and the OCI images had already been published.
>
> The cert-manager team next fixed the Helm chart and two other bugs which are listed in the "Known Issues" section below,
> and then released `v1.14.1`, which is the version that users are strongly advised to install when they upgrade to 1.14.
>
> In order to complete the stalled `v1.14.0` release,
> the Helm chart and static YAML installation files were regenerated on a team member's laptop,
> using exactly the same build scripts as are used in the automated release process,
> and using the `v1.14.1` version of the code.
> The working  `v1.14.0` Helm chart was published,
> and the working versions of the static manifest files attached to the draft `v1.14.0` GitHub release,
> and that was then published.
>
> For these reasons, users are strongly advised to skip this version and install the `v1.14.1` Helm chart instead.

##### Known Issues

-   During the release of `v1.14.0`, the Helm chart for this version was found to use the wrong OCI image for the `cainjector` Deployment,
    which caused the Helm installation to fail.
    In order to complete the release, the cert-manager team have manually updated the Helm chart for this version,
    which contains all the Helm chart fixes which are in `v1.14.1`.
    But users are strongly advised to skip this version and install the `v1.14.1` Helm chart instead.
-   A bug in cmctl namespace detection prevents it being used as a `startupapicheck` image in namespaces other than cert-manager.
-   A bug in cmctl causes `cmctl experimental install` to panic.

##### Breaking Changes

The startupapicheck job uses a new OCI image called "startupapicheck", instead of the ctl image.
If you run in an environment in which images cannot be pulled, be sure to include the new image.

The KeyUsage and BasicConstraints extensions will now be encoded as critical in the CertificateRequest's CSR blob.

##### Major Themes

##### New X.509 Features

The cert-manager Certificate resource now allows you to configure a subset of "Other Name" SANs,
which are described in the [Subject Alternative Name section of RFC 5280](https://datatracker.ietf.org/doc/html/rfc5280#section-4.2.1.6) (on page 37).

We specifically support any `otherName` type with a `UTF-8` value, such as the [User Principal Name](https://docs.venafi.com/Docs/current/TopNav/Content/Certificates/r-UEP-support-SANs.php) or [`sAMAccountName`](https://learn.microsoft.com/en-us/windows/win32/ad/naming-properties).
These are useful when issuing unique certificates for authenticating with LDAP systems such as Microsoft Active Directory.
For example you can create certificates with this block in the spec:

      otherNames:
        - oid: 1.3.6.1.4.1.311.20.2.3 # UPN OID
          utf8Value: upn@domain.local

The feature is still in alpha stage and requires you to [enable the `OtherName` feature flag in the controller and webhook components](../../installation/configuring-components.md#feature-gates).

##### New CA certificate Features

You can now specify the X.509 v3 Authority Information Accessors extension,
with URLs for certificates issued by the CA issuer.

Users can now use name constraints in CA certificates.
To know more details on name constraints check out RFC section https://datatracker.ietf.org/doc/html/rfc5280#section-4.2.1.10

##### Security

An ongoing security audit of the cert-manager code revealed some weaknesses which we have addressed in this release,
such as using more secure default settings in the HTTP servers that serve metrics, healthz and pprof endpoints.
This will help mitigate denial-of-service attacks against those important services.

All the cert-manager containers are now configured with read only root file system by default,
to prevent unexpected changes to the file system of the OCI image.

And it is now possible to configure the metrics server to use HTTPS rather than HTTP,
so that clients can verify the identity of the metrics server.

##### Other

The liveness probe of the cert-manager controller Pod is now enabled by default.

There is a new option `.spec.keystores.pkcs12.algorithms` to specify encryption and MAC algorithms for PKCS.

##### Community

Thanks again to all open-source contributors with commits in this release, including:

-   [@&#8203;ABWassim](https://github.com/ABWassim)
-   [@&#8203;JoeNorth](https://github.com/JoeNorth)
-   [@&#8203;allenmunC1](https://github.com/allenmunC1)
-   [@&#8203;asapekia](https://github.com/asapekia)
-   [@&#8203;jeremycampbell](https://github.com/jeremycampbell)
-   [@&#8203;jkroepke](https://github.com/jkroepke)
-   [@&#8203;jsoref](https://github.com/jsoref)
-   [@&#8203;lauraseidler](https://github.com/lauraseidler)
-   [@&#8203;pevidex](https://github.com/pevidex)
-   [@&#8203;phillebaba](https://github.com/phillebaba)
-   [@&#8203;snorwin](https://github.com/snorwin)
-   [@&#8203;tanujd11](https://github.com/tanujd11)
-   [@&#8203;tberreis](https://github.com/tberreis)
-   [@&#8203;vinny](https://github.com/vinny)

Thanks also to the following cert-manager maintainers for their contributions during this release:

-   [@&#8203;SgtCoDFish](https://github.com/SgtCoDFish)
-   [@&#8203;SpectralHiss](https://github.com/SpectralHiss)
-   [@&#8203;ThatsMrTalbot](https://github.com/ThatsMrTalbot)
-   [@&#8203;hawksight](https://github.com/hawksight)
-   [@&#8203;inteon](https://github.com/inteon)
-   [@&#8203;maelvls](https://github.com/maelvls)
-   [@&#8203;wallrj](https://github.com/wallrj)

Equally thanks to everyone who provided feedback, helped users and raised issues on GitHub and Slack and joined our meetings!

Thanks also to the [CNCF](https://www.cncf.io/), which provides resources and support, and to the AWS open source team for being good community members and for their maintenance of the [PrivateCA Issuer](https://github.com/cert-manager/aws-privateca-issuer).

In addition, massive thanks to [Venafi](https://www.venafi.com/) for contributing developer time and resources towards the continued maintenance of cert-manager projects.

##### Changes

##### Feature

-   ACME challenge solver Pod for HTTP01 will get a default annotation of `"cluster-autoscaler.kubernetes.io/safe-to-evict": "true"`. You can provide an annotation of `"cluster-autoscaler.kubernetes.io/safe-to-evict": "false"` in your `podTemplate` if you don't like this. ([#&#8203;6349](cert-manager/cert-manager#6349), [@&#8203;jsoref](https://github.com/jsoref))
-   Added a clock skew detector liveness probe that will force a restart in case we detect a skew between the internal monotonic clock and the system clock of more than 5 minutes.
    Also, the controller's liveness probe is now enabled by default. ([#&#8203;6328](cert-manager/cert-manager#6328), [@&#8203;inteon](https://github.com/inteon))
-   Added a new flag (--dynamic-serving-leaf-duration) that can adjust the lifetime of the dynamic leaf certificates ([#&#8203;6552](cert-manager/cert-manager#6552), [@&#8203;allenmunC1](https://github.com/allenmunC1))
-   Added support for `otherName` SANS in Certificates ([#&#8203;6404](cert-manager/cert-manager#6404), [@&#8203;SpectralHiss](https://github.com/SpectralHiss))
-   Added the option to specify the  X.509 v3 Authority Information Accessors extension CA Issuers URLs for certificates issued by the CA issuer. ([#&#8203;6486](cert-manager/cert-manager#6486), [@&#8203;jeremycampbell](https://github.com/jeremycampbell-okta))
-   Adds cert-manager's new core infrastructure initiative badge! See more details on https://www.bestpractices.dev/projects/8079 ([#&#8203;6497](cert-manager/cert-manager#6497), [@&#8203;SgtCoDFish](https://github.com/SgtCoDFish))
-   All Pods are now configured with `readOnlyRootFilesystem` by default. ([#&#8203;6453](cert-manager/cert-manager#6453), [@&#8203;wallrj](https://github.com/wallrj))
-   MAYBE BREAKING: The startupapicheck job is now handled by an entirely new container called "startupapicheck". This replaces the previous ctl container. If you run in an environment in which images cannot be pulled, be sure to include the new container. ([#&#8203;6549](cert-manager/cert-manager#6549), [@&#8203;SgtCoDFish](https://github.com/SgtCoDFish))
-   New option `.spec.keystores.pkcs12.algorithms` to specify encryption and MAC algorithms for PKCS[#&#8203;12](cert-manager/cert-manager#12) keystores. Fixes issues [#&#8203;5957](cert-manager/cert-manager#5957) and [#&#8203;6523](cert-manager/cert-manager#6523). ([#&#8203;6548](cert-manager/cert-manager#6548), [@&#8203;snorwin](https://github.com/snorwin))
-   The ACME HTTP01 solver Pod is now configured with `readOnlyRootFilesystem: true` ([#&#8203;6462](cert-manager/cert-manager#6462), [@&#8203;wallrj](https://github.com/wallrj))
-   Updates the AWS SDK for Go to 1.48.7 to support Amazon EKS Pod Identity ([#&#8203;6519](cert-manager/cert-manager#6519), [@&#8203;JoeNorth](https://github.com/JoeNorth))
-   Users can now use name constraints in CA certificates. To know more details on name constraints check out RFC section https://datatracker.ietf.org/doc/html/rfc5280#section-4.2.1.10 ([#&#8203;6500](cert-manager/cert-manager#6500), [@&#8203;tanujd11](https://github.com/tanujd11))
-   ⚠️ potentially breaking ⚠️: The KeyUsage and BasicConstraints extensions will now be encoded as critical in the CertificateRequest's CSR blob. ([#&#8203;6053](cert-manager/cert-manager#6053), [@&#8203;inteon](https://github.com/inteon))
-   Add TLS support to the metrics endpoint through either a certificate file or through dynamically issued certificates ([#&#8203;6574](cert-manager/cert-manager#6574), [@&#8203;ThatsMrTalbot](https://github.com/ThatsMrTalbot))
-   Helm Chart: allow changing the default Deployment `revisionHistoryLimit` ([#&#8203;6248](cert-manager/cert-manager#6248), [@&#8203;tberreis](https://github.com/tberreis))
-   Security: Limit the size of the response body read from HTTP requests by cert-manager. ([#&#8203;6619](cert-manager/cert-manager#6619), [@&#8203;ThatsMrTalbot](https://github.com/ThatsMrTalbot))
-   Support custom `spec.namespaceSelector` for webhooks ([#&#8203;6638](cert-manager/cert-manager#6638), [@&#8203;jkroepke](https://github.com/jkroepke))

##### Bug or Regression

-   BUGFIX\[helm]: Fix issue where webhook feature gates were only set if controller feature gates are set. ([#&#8203;6380](cert-manager/cert-manager#6380), [@&#8203;asapekia](https://github.com/asapekia))
-   Controller ConfigMap is now created only if `.Values.config` is set. ([#&#8203;6357](cert-manager/cert-manager#6357), [@&#8203;ABWassim](https://github.com/ABWassim))
-   Fix runaway bug caused by multiple Certificate resources that point to the same Secret resource. ([#&#8203;6406](cert-manager/cert-manager#6406), [@&#8203;inteon](https://github.com/inteon))
-   Fix(helm): templating of required value in controller and webhook ConfigMap resources ([#&#8203;6435](cert-manager/cert-manager#6435), [@&#8203;ABWassim](https://github.com/ABWassim))
-   Fixed a webhook validation error message when the key algorithm was invalid. ([#&#8203;6571](cert-manager/cert-manager#6571), [@&#8203;pevidex](https://github.com/pevidex))
-   Fixed error messaging when setting up vault issuer ([#&#8203;6433](cert-manager/cert-manager#6433), [@&#8203;vinny](https://github.com/vinny-sabatini))
-   `GHSA-vgf6-pvf4-34rq`: The webhook server now returns HTTP error 413 (Content Too Large) for requests with body size `>= 3MiB`. This is to mitigate DoS attacks that attempt to crash the webhook process by sending large requests that exceed the available memory.
    The webhook server now returns HTTP error 400 (Bad Request) if the request contains an empty body.
    The webhook server now returns HTTP error 500 (Internal Server Error) rather than crashing, if the code panics while handling a request. ([#&#8203;6498](cert-manager/cert-manager#6498), [@&#8203;inteon](https://github.com/inteon))
-   Increase the default webhook timeout to its maximum value of 30 seconds, so that the underlying timeout error message has more chance of being returned to the end user. ([#&#8203;6488](cert-manager/cert-manager#6488), [@&#8203;wallrj](https://github.com/wallrj))
-   Listeners that do not support TLS on Gateway resources will now not raise `BadConfig` warnings anymore ([#&#8203;6347](cert-manager/cert-manager#6347), [@&#8203;lauraseidler](https://github.com/lauraseidler))
-   Mitigate potential Slowloris attacks by setting `ReadHeaderTimeout` in all `http.Server` instances ([#&#8203;6534](cert-manager/cert-manager#6534), [@&#8203;wallrj](https://github.com/wallrj))
-   The Venafi issuer now properly resets the certificate and should no longer get stuck with `WebSDK CertRequest Module Requested Certificate` or `This certificate cannot be processed while it is in an error state. Fix any errors, and then click Retry.`. ([#&#8203;6398](cert-manager/cert-manager#6398), [@&#8203;maelvls](https://github.com/maelvls))
-   Update experimental install and uninstall commands to have flag parity with the rest of the CLI ([#&#8203;6562](cert-manager/cert-manager#6562), [@&#8203;ThatsMrTalbot](https://github.com/ThatsMrTalbot))
-   Webhook ConfigMap if now created only if `.Values.webhook.config` is set. ([#&#8203;6360](cert-manager/cert-manager#6360), [@&#8203;ABWassim](https://github.com/ABWassim))
-   BUGFIX: Ensure `otherName` SAN changes in Certificate resources trigger re-issuance. ([#&#8203;6620](cert-manager/cert-manager#6620), [@&#8203;SpectralHiss](https://github.com/SpectralHiss))
-   Bugfix: Publish the `startupapicheck` image to `quay.io` ([#&#8203;6609](cert-manager/cert-manager#6609), [@&#8203;wallrj](https://github.com/wallrj))

##### Other (Cleanup or Flake)

-   Cert-manager is now built with Go 1.21.5 ([#&#8203;6545](cert-manager/cert-manager#6545), [@&#8203;wallrj](https://github.com/wallrj))
-   Bump Go to `1.21.3` to address `CVE-2023-39325`. Also bumps base images. ([#&#8203;6410](cert-manager/cert-manager#6410), [@&#8203;SgtCoDFish](https://github.com/SgtCoDFish))
-   Bump `golang.org/x/net v0.15.0 => v0.17.0` as part of addressing `CVE-2023-44487` / `CVE-2023-39325` ([#&#8203;6427](cert-manager/cert-manager#6427), [@&#8203;SgtCoDFish](https://github.com/SgtCoDFish))
-   Check code for unintended use of `crypto/md5`, a weak cryptographic primitive; using `golangci-lint` / `gosec` (G501). ([#&#8203;6581](cert-manager/cert-manager#6581), [@&#8203;wallrj](https://github.com/wallrj))
-   Check code for unintended use of `crypto/sha1`, a weak cryptographic primitive; using `golangci-lint` / `gosec` (G505). ([#&#8203;6579](cert-manager/cert-manager#6579), [@&#8203;wallrj](https://github.com/wallrj))
-   Check code for unintended use of weak random number generator (`math/rand` instead of `crypto/rand`); using `golangci-lint` / `gosec` (G404). ([#&#8203;6582](cert-manager/cert-manager#6582), [@&#8203;wallrj](https://github.com/wallrj))
-   Cleanup: Restrict MutatingWebhookConfiguration to only CertificateRequest resources ([#&#8203;6311](cert-manager/cert-manager#6311), [@&#8203;hawksight](https://github.com/hawksight))
-   Deprecated `pkg/util.RandStringRunes` and `pkg/controller/test.RandStringBytes`. Use `k8s.io/apimachinery/pkg/util/rand.String` instead. ([#&#8203;6585](cert-manager/cert-manager#6585), [@&#8203;wallrj](https://github.com/wallrj))
-   Enabled verbose logging in startupapicheck by default, so that if it fails, users can know exactly what caused the failure. ([#&#8203;6495](cert-manager/cert-manager#6495), [@&#8203;wallrj](https://github.com/wallrj))
-   Fix gosec G601: Implicit memory aliasing of items from a range statement ([#&#8203;6551](cert-manager/cert-manager#6551), [@&#8203;wallrj](https://github.com/wallrj))
-   Fix handling of serial numbers in literal certificate subjects. Previously a serial number could be specified in `subject.serialNumber` while using a literal certificate subject. This was a mistake and has been fixed. ([#&#8203;6533](cert-manager/cert-manager#6533), [@&#8203;inteon](https://github.com/inteon))
-   The end-to-end tests can now test the cert-manager Vault Issuer on an OpenShift cluster. ([#&#8203;6391](cert-manager/cert-manager#6391), [@&#8203;wallrj](https://github.com/wallrj))
-   Update cert-manager's distroless base images from Debian 11 to Debian 12. This should have no practical effects on users. ([#&#8203;6583](cert-manager/cert-manager#6583), [@&#8203;inteon](https://github.com/inteon))
-   Updated all code using GatewayAPI to use the now GA v1 APIs ([#&#8203;6559](cert-manager/cert-manager#6559), [@&#8203;ThatsMrTalbot](https://github.com/ThatsMrTalbot))
-   Upgrade Go from 1.20.7 to 1.20.8. ([#&#8203;6369](cert-manager/cert-manager#6369), [@&#8203;inteon](https://github.com/inteon))
-   Upgrade `github.com/emicklei/go-restful/v3` to `v3.11.0` because `v3.10.2` is labeled as "DO NOT USE". ([#&#8203;6366](cert-manager/cert-manager#6366), [@&#8203;inteon](https://github.com/inteon))
-   Use the new generic `sets.Set` type in place of the deprecated `sets.String`. ([#&#8203;6586](cert-manager/cert-manager#6586), [@&#8203;wallrj](https://github.com/wallrj))
-   cert-manager is now built with Go `v1.21.6` ([#&#8203;6628](cert-manager/cert-manager#6628), [@&#8203;SgtCoDFish](https://github.com/SgtCoDFish))
-   Update the Azure SDK and remove deprecated `autorest` dependency ([#&#8203;5452](cert-manager/cert-manager#5452), [@&#8203;phillebaba](https://github.com/phillebaba))
-   The cert-manager E2E tests can now be run on Kubernetes 1.29 ([#&#8203;6641](cert-manager/cert-manager#6641), [@&#8203;wallrj](https://github.com/wallrj))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xNjUuMCIsInVwZGF0ZWRJblZlciI6IjM3LjE2NS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Reviewed-on: https://git.home/nrdufour/home-ops/pulls/358
Co-authored-by: Renovate <renovate@ptinem.io>
Co-committed-by: Renovate <renovate@ptinem.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/testing Issues relating to testing dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants