Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUGFIX: Stricter CertificateRequest CSR webhook validation #6182

Conversation

inteon
Copy link
Member

@inteon inteon commented Jun 26, 2023

Pull Request Motivation

While creating #6056, we realised that the current CertificateRequest CSR webhook validation logic is too permissive:
If CertificateRequest.Usages is empty and the KeyUsages are only set as an extension in the CSR blob, the validation webhook allows this CertificateRequest to be created. However, some solutions (like approver-policy) depend on the Usages always being defined in the Usages array.

This is a quick summary of the current webhook behavior vs the new behavior:

situation before after
usages in CertReq & CSR blob error if != error if !=
usages only in CertReq allow (use CertReq usages) allow (use CertReq usages)
usages only in CSR blob allow (use CSR blob usages) error if CSR blob usages != default usages
no usages defined allow (use default usages) allow (use default usages)

Important note: The current webhook behavior differs from our assumptions made in approver-policy & our assumptions made in our Certificate Template logic (https://github.com/cert-manager/cert-manager/blob/master/pkg/util/pki/certificatetemplate.go#L206).

Making the validation webhook stricter is always a breaking change, however I think it is justified because it makes cert-manager more secure (after this change, you can rely purely on the Usages array in the CertReq to make policy decisions). Also, there is no existing implementation that uses the CSR blob as sole source of truth for the usages. (see examples below)
[OK] csi-driver sets Usages only on CertReq: https://github.com/cert-manager/csi-driver/blob/ebe0181784d79b8514bb77a5e516ea978da4c6bd/pkg/requestgen/generator.go#L76-L95
[OK] csi-driver-spiffe sets Usages only on CertReq: https://github.com/cert-manager/csi-driver-spiffe/blob/main/internal/csi/driver/driver.go#L274-L289
[OK] istio-csr sets Usages both on CertReq and CSR blob: https://github.com/cert-manager/istio-csr/blob/main/pkg/server/server.go#L180

Kind

/kind bug

Release Note

⚠️ possibly breaking: Webhook validation of CertificateRequest resources is stricter now: all KeyUsages and ExtendedKeyUsages must be defined directly in the CertificateRequest resource, the encoded CSR can never contain more usages that defined there.

Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
…ray should always be the source of truth)

Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
@jetstack-bot jetstack-bot added kind/bug Categorizes issue or PR as related to a bug. release-note Denotes a PR that will be considered when it comes time to generate release notes. dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. area/testing Issues relating to testing approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jun 26, 2023
@SgtCoDFish
Copy link
Member

SgtCoDFish commented Jun 27, 2023

My 2c after thinking about this and sleeping on it:

I think this sort of thing is a worthwhile change to make and it's reasonable to have it on by default because, as you say, there's a security improvement to be had from having a single source of truth for any given element of a certificate.

There are some caveats to that, though:

  1. I don't tend to think that having the CertificateRequest be the single source of truth is the best option. The CSR is a better source of truth in my mind because:
    1. It's signed, and
    2. It's infinitely more customisable than the CertificateRequest - people can put whatever they like in a CSR and it's not a huge change to cert-manager to allow, say, custom extensions or whatever in the future. That means that for future additions I'd think we'd want to add them to the CSR and so it'd be nice to do that here too.
  2. That said, I suspect that with tools like approver-policy relying on the Usages today, we can't make the CSR the source of truth for this field and we'd have to wait for v2 (if ever), so maybe 1 is moot
  3. I think making a breaking change and then saying "there's no remediation if this breaks you except to change all your workflows" is much more disruptive than adding some sort of flag. Could we add --insecure-allow-csr-usage-definitions or similar to the webhook, so that people who are broken by this can choose to preserve the old behaviour? I don't think - from looking at this change - that that would be a huge burden for us?

@jetstack-bot jetstack-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 28, 2023
…trict CSR validation

Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
@inteon inteon force-pushed the stricter_certificaterequest_csr_webhook_validation branch from 46e9c76 to 2f56c3c Compare June 28, 2023 09:11
@inteon
Copy link
Member Author

inteon commented Jun 28, 2023

@SgtCoDFish I added the DontAllowInsecureCSRUsageDefinition feature gate, which can be used to disable this new behavior (when set to false).
The gate is enabled by default however (default = true).

Copy link
Member

@SgtCoDFish SgtCoDFish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve
/hold

Adding a hold because of my suggestion - but I think it's fine to merge as is!

Comment on lines +51 to +56
// Owner (responsible for graduating feature through to GA): @inteon
// GA: v1.13
// DontAllowInsecureCSRUsageDefinition will prevent the webhook from allowing
// CertificateRequest's usages to be only defined in the CSR, while leaving
// the usages field empty.
DontAllowInsecureCSRUsageDefinition featuregate.Feature = "DontAllowInsecureCSRUsageDefinition"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: I don't think this flag really matches my understanding of what we want a "feature gate" to be.

My understanding was that a feature gate was something we were concerned might break or need to change in the future.

I don't really see a huge problem here with this just being a regular CLI flag. I suppose a feature gate makes it easier to remove this in the future, but I don't really see a pressing need to remove this option.

It doesn't really matter though!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that we would like to remove this feature flag in an upcoming release if no one complains that this change causes problems for them. That is why I added this as a feature flag.
I think you only want to be less strict in case you have a cert-manager integration that "incorrectly" uses the API.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future, I would like to make some other improvements to our validation webhook that might be a bit more difficult to implement with this feature flag (not sure). eg. #6056

@jetstack-bot jetstack-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 29, 2023
@jetstack-bot jetstack-bot added the lgtm Indicates that a PR is ready to be merged. label Jun 29, 2023
@jetstack-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: inteon, SgtCoDFish

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@inteon
Copy link
Member Author

inteon commented Jun 29, 2023

Thanks @SgtCoDFish
/unhold see #6182 (comment)

@jetstack-bot jetstack-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 29, 2023
@jetstack-bot jetstack-bot merged commit e66a92a into cert-manager:master Jun 29, 2023
6 checks passed
nrdufour added a commit to nrdufour/home-ops that referenced this pull request Sep 12, 2023
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [cert-manager](https://github.com/cert-manager/cert-manager) | minor | `v1.12.4` -> `v1.13.0` |

---

### Release Notes

<details>
<summary>cert-manager/cert-manager (cert-manager)</summary>

### [`v1.13.0`](https://github.com/cert-manager/cert-manager/releases/tag/v1.13.0)

[Compare Source](cert-manager/cert-manager@v1.12.4...v1.13.0)

cert-manager is the easiest way to automatically manage certificates in Kubernetes and OpenShift clusters.

This is the 1.13 release of cert-manager!

#### Community

Welcome to these new cert-manager members (more info - cert-manager/cert-manager#6260):
[@&#8203;jsoref](https://github.com/jsoref)
[@&#8203;FlorianLiebhart](https://github.com/FlorianLiebhart)
[@&#8203;hawksight](https://github.com/hawksight)
[@&#8203;erikgb](https://github.com/erikgb)

Thanks again to all open-source contributors with commits in this release, including:
[@&#8203;AcidLeroy](https://github.com/AcidLeroy)
[@&#8203;FlorianLiebhart](https://github.com/FlorianLiebhart)
[@&#8203;lucacome](https://github.com/lucacome)
[@&#8203;cypres](https://github.com/cypres)
[@&#8203;erikgb](https://github.com/erikgb)
[@&#8203;ubergesundheit](https://github.com/ubergesundheit)
[@&#8203;jkroepke](https://github.com/jkroepke)
[@&#8203;jsoref](https://github.com/jsoref)
[@&#8203;gdvalle](https://github.com/gdvalle)
[@&#8203;rouke-broersma](https://github.com/rouke-broersma)
[@&#8203;schrodit](https://github.com/schrodit)
[@&#8203;zhangzhiqiangcs](https://github.com/zhangzhiqiangcs)
[@&#8203;arukiidou](https://github.com/arukiidou)
[@&#8203;hawksight](https://github.com/hawksight)
[@&#8203;Richardds](https://github.com/Richardds)
[@&#8203;kahirokunn](https://github.com/kahirokunn)

Thanks also to the following cert-manager maintainers for their contributions during this release:
[@&#8203;SgtCoDFish](https://github.com/SgtCoDFish)
[@&#8203;maelvls](https://github.com/maelvls)
[@&#8203;irbekrm](https://github.com/irbekrm)
[@&#8203;inteon](https://github.com/inteon)

Equally thanks to everyone who provided feedback, helped users and raised issues on Github and Slack and joined our meetings!

Special thanks to [@&#8203;AcidLeroy](https://github.com/AcidLeroy) for adding "load options from a versioned config file" support for the cert-manager controller! This has been on our wishlist for a very long time. (see cert-manager/cert-manager#5337)

Also, thanks a lot to [@&#8203;FlorianLiebhart](https://github.com/FlorianLiebhart) for adding support for DNS over HTTPS for the ACME DNS self-check. This is very useful in case all traffic must be HTTP(S) trafic, eg. when using a HTTPS_PROXY. (see cert-manager/cert-manager#5003)

Thanks also to the [CNCF](https://www.cncf.io/), which provides resources and support, and to the AWS open source team for being good community members and for their maintenance of the [PrivateCA Issuer](https://github.com/cert-manager/aws-privateca-issuer).

In addition, massive thanks to [Venafi](https://www.venafi.com/) for contributing developer time and resources towards the continued maintenance of cert-manager projects.

#### Changes since v1.12.0

##### Feature

-   Add support for logging options to webhook config file. ([#&#8203;6243](cert-manager/cert-manager#6243), [@&#8203;inteon](https://github.com/inteon))
-   Add view permissions to the well-known (Openshift) user-facing `cluster-reader` aggregated cluster role ([#&#8203;6241](cert-manager/cert-manager#6241), [@&#8203;erikgb](https://github.com/erikgb))
-   Certificate Shim: distinguish dns names and ip address in certificate ([#&#8203;6267](cert-manager/cert-manager#6267), [@&#8203;zhangzhiqiangcs](https://github.com/zhangzhiqiangcs))
-   Cmctl can now be imported by third parties. ([#&#8203;6049](cert-manager/cert-manager#6049), [@&#8203;SgtCoDFish](https://github.com/SgtCoDFish))
-   Make `enableServiceLinks` configurable for all Deployments and `startupapicheck` Job in Helm chart. ([#&#8203;6292](cert-manager/cert-manager#6292), [@&#8203;ubergesundheit](https://github.com/ubergesundheit))
-   Promoted the StableCertificateRequestName and SecretsFilteredCaching feature gates to Beta (enabled by default). ([#&#8203;6298](cert-manager/cert-manager#6298), [@&#8203;inteon](https://github.com/inteon))
-   The cert-manager controller options are now configurable using a configuration file. ([#&#8203;5337](cert-manager/cert-manager#5337), [@&#8203;AcidLeroy](https://github.com/AcidLeroy))
-   The pki CertificateTemplate functions now perform validation of the CSR blob, making sure we sign a Certificate that matches the IsCA and (Extended)KeyUsages that are defined in the CertificateRequest resource. ([#&#8203;6199](cert-manager/cert-manager#6199), [@&#8203;inteon](https://github.com/inteon))
-   \[helm] Add prometheus.servicemonitor.endpointAdditionalProperties to define additional properties on a ServiceMonitor endpoint, e.g. relabelings ([#&#8203;6110](cert-manager/cert-manager#6110), [@&#8203;jkroepke](https://github.com/jkroepke))

##### Design

-   DNS over HTTPS (DoH) is now possible for doing the self-checks during the ACME verification.
    The DNS check method to be used is controlled through the command line flag: `--dns01-recursive-nameservers-only=true` in combination with `--dns01-recursive-nameservers=https://<DoH-endpoint>` (e.g. `https://8.8.8.8/dns-query`). It keeps using DNS lookup as a default method. ([#&#8203;5003](cert-manager/cert-manager#5003), [@&#8203;FlorianLiebhart](https://github.com/FlorianLiebhart))

##### Bug or Regression

-   Allow overriding default pdb .minAvailable with .maxUnavailable without setting .minAvailable to null ([#&#8203;6087](cert-manager/cert-manager#6087), [@&#8203;rouke-broersma](https://github.com/rouke-broersma))
-   BUGFIX: `cmctl check api --wait 0` exited without output and exit code 1; we now make sure we perform the API check at least once and return with the correct error code ([#&#8203;6109](cert-manager/cert-manager#6109), [@&#8203;inteon](https://github.com/inteon))
-   BUGFIX: the issuer and certificate-name annotations on a Secret were incorrectly updated when other fields are changed. ([#&#8203;6147](cert-manager/cert-manager#6147), [@&#8203;inteon](https://github.com/inteon))
-   BUGFIX\[cainjector]: 1-character bug was causing invalid log messages and a memory leak ([#&#8203;6232](cert-manager/cert-manager#6232), [@&#8203;inteon](https://github.com/inteon))
-   Fix CloudDNS issuers stuck in propagation check, when multiple instances are issuing for the same FQDN ([#&#8203;6088](cert-manager/cert-manager#6088), [@&#8203;cypres](https://github.com/cypres))
-   Fix indentation of Webhook NetworkPolicy matchLabels in helm chart. ([#&#8203;6220](cert-manager/cert-manager#6220), [@&#8203;ubergesundheit](https://github.com/ubergesundheit))
-   Fixed Cloudflare DNS01 challenge provider race condition when validating multiple domains ([#&#8203;6191](cert-manager/cert-manager#6191), [@&#8203;Richardds](https://github.com/Richardds))
-   Fixes a bug where webhook was pulling in controller's feature gates.
    ⚠️  ⚠️ BREAKING ⚠️ ⚠️ : If you deploy cert-manager using helm and have `.featureGates` value set, the features defined there will no longer be passed to cert-manager webhook, only to cert-manager controller. Use `webhook.featureGates` field instead to define features to be enabled on webhook.
    **Potentially breaking**: If you were, for some reason, passing cert-manager controller's features to webhook's `--feature-gates` flag, this will now break (unless the webhook actually has a feature by that name). ([#&#8203;6093](cert-manager/cert-manager#6093), [@&#8203;irbekrm](https://github.com/irbekrm))
-   Fixes an issue where cert-manager would incorrectly reject two IP addresses as being unequal when they should have compared equal. This would be most noticeable when using an IPv6 address which doesn't match how Go's `net.IP.String()` function would have printed that address. ([#&#8203;6293](cert-manager/cert-manager#6293), [@&#8203;SgtCoDFish](https://github.com/SgtCoDFish))
-   We disabled the `enableServiceLinks` option for our ACME http solver pods, because the option caused the pod to be in a crash loop in a cluster with lot of services. ([#&#8203;6143](cert-manager/cert-manager#6143), [@&#8203;schrodit](https://github.com/schrodit))
-   ⚠️ possibly breaking: Webhook validation of CertificateRequest resources is stricter now: all KeyUsages and ExtendedKeyUsages must be defined directly in the CertificateRequest resource, the encoded CSR can never contain more usages that defined there. ([#&#8203;6182](cert-manager/cert-manager#6182), [@&#8203;inteon](https://github.com/inteon))

##### Other (Cleanup or Flake)

-   A subset of the klogs flags have been deprecated and will be removed in the future. ([#&#8203;5879](cert-manager/cert-manager#5879), [@&#8203;maelvls](https://github.com/maelvls))
-   All service links in helm chart deployments have been disabled. ([#&#8203;6144](cert-manager/cert-manager#6144), [@&#8203;schrodit](https://github.com/schrodit))
-   Cert-manager will now re-issue a certificate if the public key in the latest CertificateRequest resource linked to a Certificate resource does not match the public key of the key encoded in the Secret linked to that Certificate resource ([#&#8203;6168](cert-manager/cert-manager#6168), [@&#8203;inteon](https://github.com/inteon))
-   Chore: When hostNetwork is enabled, dnsPolicy is now set to ClusterFirstWithHostNet. ([#&#8203;6156](cert-manager/cert-manager#6156), [@&#8203;kahirokunn](https://github.com/kahirokunn))
-   Cleanup the controller configfile structure by introducing sub-structs. ([#&#8203;6242](cert-manager/cert-manager#6242), [@&#8203;inteon](https://github.com/inteon))
-   Don't run API Priority and Fairness controller in webhook's extension apiserver ([#&#8203;6085](cert-manager/cert-manager#6085), [@&#8203;irbekrm](https://github.com/irbekrm))
-   Helm: Add apache 2.0 license annotation ([#&#8203;6225](cert-manager/cert-manager#6225), [@&#8203;arukiidou](https://github.com/arukiidou))
-   Make apis/acme/v1/ACMEIssuer.PreferredChain optional in JSON serialization. ([#&#8203;6034](cert-manager/cert-manager#6034), [@&#8203;gdvalle](https://github.com/gdvalle))
-   The SecretPostIssuancePolicyChain now also makes sure that the `cert-manager.io/common-name`, `cert-manager.io/alt-names`, ... annotations on Secrets are kept at their correct value. ([#&#8203;6176](cert-manager/cert-manager#6176), [@&#8203;inteon](https://github.com/inteon))
-   The cmctl logging has been improved and support for json logging has been added. ([#&#8203;6247](cert-manager/cert-manager#6247), [@&#8203;inteon](https://github.com/inteon))
-   Updates Kubernetes libraries to `v0.27.2`. ([#&#8203;6077](cert-manager/cert-manager#6077), [@&#8203;lucacome](https://github.com/lucacome))
-   Updates Kubernetes libraries to `v0.27.4`. ([#&#8203;6227](cert-manager/cert-manager#6227), [@&#8203;lucacome](https://github.com/lucacome))
-   We now only check that the issuer name, kind and group annotations on a Secret match in case those annotations are set. ([#&#8203;6152](cert-manager/cert-manager#6152), [@&#8203;inteon](https://github.com/inteon))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi4yMy4yIiwidXBkYXRlZEluVmVyIjoiMzYuMjMuMiIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Reviewed-on: https://git.home/nrdufour/home-ops/pulls/84
Co-authored-by: Renovate <renovate@ptinem.io>
Co-committed-by: Renovate <renovate@ptinem.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/testing Issues relating to testing dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants