Enable workload sharding in prow

[0xmichalis]

It should be possible to shard the workload in prow between multiple controllers for the same agent. The granularity level needs to be at the job level but we should also support sharding per repositories.

Use-cases:

• canarying prow changes in repositories other than the core ones
• we run two Jenkins masters and a custom proxy in between Jenkins and prow and need to get rid of the proxy because it's making debugging hard. I think that the best approach would be to shard based on the jobs so we can effectively shard between the two masters.

/area prow

[spxtr]

Maybe pass a label (list?) as an argument to the controller and have it list only those prowjobs and pods?

[0xmichalis]

Yes, that's what I was thinking. We will need to either include or exclude labels in controllers.

[0xmichalis]

/assign

[munnerz]

Regarding doing this after #1986, I think we can make use of the new filtering made possible with kubernetes/kubernetes#54660 when constructing a SharedInformerFactory. This should let us limit the results to only those that match a label selector.

[0xmichalis]

@munnerz we already fixed sharding. If we ever switch to use informers, we should do it in a way that wouldn't break sharding.