ceph: Add RBAC in helm chart for mgr to create service monitor

[sathieu]

Description of your changes:

Same as #8118, for Helm chart.

Which issue is resolved by this Pull Request:
Resolves #

Checklist:

• Commit Message Formatting: Commit titles and messages follow guidelines in the developer guide.
• Skip Tests for Docs: Add the flag for skipping the build if this is only a documentation change. See here for the flag.
• Skip Unrelated Tests: Add a flag to run tests for a specific storage provider. See test options.
• Reviewed the developer guide on Submitting a Pull Request
• Documentation has been updated, if necessary.
• Unit tests have been added, if necessary.
• Integration tests have been added, if necessary.
• Pending release notes updated with breaking and/or notable changes, if necessary.
• Upgrade from previous release is tested and upgrade user guide is updated, if necessary.
• Code generation (make codegen) has been run to update object specifications, if necessary.

[BlaineEXE]

Thank you @sathieu for contributing this. We removed recently this bit of code in the operator helm chart because we don't want to strongly tie installation of Rook to Prometheus.

In our example manifests, we have deploy/examples/monitoring/rbac.yaml which deploys the RBAC needed for Prometheus monitoring for users who manually install, and we have the same optional-ness for users of the rook-ceph-cluster Helm chart, which can optionally deploy the monitoring RBAC.

Given that not all users are using the rook-ceph-cluster chart, I have proposed a different solution that will optionally install the required helm resources in the rook-ceph chart. See here: #9388

@sathieu would you care to test that new PR's changes to make sure your workflow is working with it?

[BlaineEXE]

Requesting changes while we evaluate a different solution.

[BlaineEXE]

Monitoring should not be enabled for all users by default. For our releases that go out to hundreds of users, we'd like this to be false by default, and for those who want monitoring, they enable it via monitoring.enabled.

[BlaineEXE]

Also, please change the subject to helm: ... for our feature tracking. ceph: ... is a bit too generic.

[BlaineEXE]

Closing now that #9388 is merged.