Move otel operator CRDs to templates

[swiatekm-sumo]

Partially resolves #1167. See the issue for more details. I ended up patching both the webhook name and the certificate name in addition to the namespace in the CRD.

I verified the upgrade procedure manually.

[swiatekm-sumo]

Looks like the failing test is flaky, it passes locally for me.

[TylerHelmuth]

Similar to how managing the CRDs via the crd directory is optional I think installing the CRDs via the template should be optional as well. We should add a new section to the values.yaml that allows controlling when the templated CRDs are installed.

[TylerHelmuth]

You will need to bump the minor version of the chart and regenerate the examples via make generate-examples CHARTS=opentelemetry-operator.

[swiatekm-sumo]

Similar to how managing the CRDs via the crd directory is optional I think installing the CRDs via the template should be optional as well. We should add a new section to the values.yaml that allows controlling when the templated CRDs are installed.

Does a top-level installCrds option sound good? Or do we want something more elaborate?

[TylerHelmuth]

Since we're allowing templating, it feels possible someone will want to set a specific value, so I think a section called crds or installCRDs with an enabled field is good.

[dmitryax]

LGTM

[jaronoff97]

+1 to tyler's recommendation

[swiatekm-sumo]

Added a crds section to the values file, with a create: true option. This makes it consistent with other create-stuff options that currently exist.

It's increasingly annoying to automatically convert CRDs published by the operator into the form the Helm Chart wants. I added a make function to make this easier, but at some point we may want to look for a more holistic solution here.

@TylerHelmuth can you have another look? Also @pavolloffay.

[jaronoff97]

🎉

[swiatekm-sumo]

I think something I'm missing here is that this change only works with cert-manager enabled. I should inject the caBundle directly when using a Secret. Please don't merge this until then, I'll get to it tomorrow morning.

On that note, we should really run the e2e tests with cert-manager disabled.

[TylerHelmuth]

Currently the chart has a cert-manager requirement by default, but supports other options. We definitely need the CRDs to work without the cert manager like they were before. I'm ok with the CI continuing to use cert manager for now since that is our default requirement

[swiatekm-sumo]

Thinking about it a bit more, that change isn't necessary in this PR, as there's no conversion webhooks defined, and the CRDs don't need a caBundle. So we can merge this as-is and solve this problem in the actual upgrade do 0.99.0.

[swiatekm-sumo]

I've added running e2e tests without certmanager to the CI. We can remove it later if you want, but I'd like to have confidence everything works right in both cases for the CRD migration.