Create a separate registry for work-queue metrics

[sonamkshenoy]

Hi all!

We are using custom controllers to manage our custom resources, and wish to export the workqueue metrics to Prometheus.
This is done by using the workqueue.SetProvider() provided by kubernetes.io/client-go.
When we try to register our metrics using this function in our init(), this doesn't take effect, as the init() function being used in the metrics package of this repo seems to making the call to SetProvider() first.

We can make use of this registry that you provide. It however comes with a whole lot of other metrics which we do not wish to export, such as - leader election, latency, request size, response size and other client metrics. And since they're internal we aren't able to unregister them.

Wonder if we can either:

1. [The best option in my opinion] Create a separate registry for just workqueue metrics in this repo and not add the other metrics to it, so that we can use it in our project. We can then combine all those metrics along with the non-workqueue related ones in a separate registry, thus unaffecting the previous behaviour.
2. Not have the init() function altogether in the metrics.go file. Instead provide those functionalities as a separate function, so that we users can call it in our init(). This way, if we choose to, we can create our own metrics and call SetProvider() ourselves.
3. Any other option which might not have been covered above...
4. Or is there a way we can tackle this on our side without having to make any changes here?

I'm up for working on this issue.

[troy0820]

/kind support feature

[The best option in my opinion] Create a separate registry for just workqueue metrics in this repo and not add the other metrics to it, so that we can use it in our project. We can then combine all those metrics along with the non-workqueue related ones in a separate registry, thus unaffecting the previous behaviour.

I think this is a reasonable request. Do you mean that the defined metrics can then be registered to this "new" registry and pick the metrics you want to export rather than having all in one? Such as workqueue, etc?

And allow the user to register the metrics they want to expose?

[sonamkshenoy]

Yep!
Something like: #2671

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale