Consider shipping multiple go modules.

[joeblubaugh]

Proposal

Ship multiple go modules in this repository. For example, github.com/prometheus/prometheus/promql (or simply github.com/prometheus/promql) could be a module, as could github.com/prometheus/discovery. Many sub-directories in the project don't have direct dependencies on each other, but depend on github.com/prometheus/prometheus/common and github.com/prometheus/prometheus/util.

Use case. Why is this important?

My primary issue is that the whole prometheus module lists a requirement for k8s.io/client-go v0.20.0, but this is only needed by the discovery/ path. I needn't use the kubernetes discovery in my existing project, but I would like to use a recent version of the promql package. I do use the kubernetes API in other parts of my project, and I need it pinned to an earlier version of the code than prometheus requires. MVS always picks the version required by the prometheus module. I therefore can't build my project in a way that's compatible with my runtime while including the prometheus/prometheus module.

I can work around this by copying the part of the source tree that I need into its own directory, or shadowing the repo, deleting all the code I don't want and updating the go.mod file, but it's not an ideal long-term solution.

[roidelapluie]

Thank you for your interest. While having multiple modules would slow down prometheus development and raise the barrier to entry, this is something that will be discussed in our developer summit, in the coming months.

For your specific dependency conflict, it should be fixed in the coming release of go (go 1.17): golang/go#36460
Can you hold a few month more with your current workflow? Thanks.

[roidelapluie]

(note: we also experimented with https://github.com/modularise but it seems not to work anymore)

[joeblubaugh]

Thanks - I think a discussion at the developer summit sounds like a great way to consider the issue. It's certainly not a slam-dunk do-it-now proposal. I think our workaround will be sufficient for my team for now.

I'm finding this design document a little hard to follow: https://go.googlesource.com/proposal/+/master/design/36460-lazy-module-loading.md
But I do believe that as long as we don't import the discovery package, we won't have the prometheus transitive dependency on k8s and it won't contribute to the k8s version selected by us.

[roidelapluie]

Yes that is my understanding.

[beorn7]

Last week's dev summit didn't get far enough through the agenda to cover this, but I expect it to be handled soon at one of the upcoming summits (happen monthly at the moment).

Relevant previous discussions:
https://groups.google.com/g/prometheus-developers/c/F1Vp0rLk3TQ/m/TyF2WxlkBgAJ
https://groups.google.com/g/prometheus-developers/c/DY88o6yOg28/m/Bzc7E0x9BgAJ