Reject PrometheusRule objects that generate > 512KiB configmap

[simonpasquier]

          @simonpasquier the title of the issue was generic decently generic, I'm happy to open a new issue as I realize my proposed question is not directly related.

---

The error is obvious:

level=error ts=2024-03-21T13:51:21.598484221Z caller=klog.go:116 component=k8s_client_runtime func=ErrorDepth msg="sync \"monitoring/kube-prometheus-stack-prometheus\" failed: failed to make rules ConfigMaps: rule file 'monitoring-kube-prometheus-stack-metrics-catalog-rules-aws-836e95d3-92a2-493e-bee9-6a33bb2cc55b.yaml' is too large for a single Kubernetes ConfigMap"

In this case, the ruleset that the CRD is trying to create simply exceeds the 1MB limit for ConfigMaps. So I know the problem, but I'm hoping we can move this error to prior to the CRD accepting the change and thus failing the deployment. Otherwise, I don't know things are wrong until my attempt at looking at data showcases that I'm missing rules.

Originally posted by @jtslear in #5085 (comment)

[simonpasquier]

@jtslear IIUC correctly your problem, you've got a PrometheusRule which by itself is above the 1MB limit?
In this case, yes it would be a good idea for the validating webhook service to reject the object in the first place.

[yp969803]

/assign