Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix labels for Service Monitors #2878

Merged
merged 24 commits into from May 20, 2024
Merged

Conversation

iblancasa
Copy link
Contributor

@iblancasa iblancasa commented Apr 18, 2024

Description:: fix the labels for the Service Monitor integration. Exclude the headless Service from the integration with SM.

Resolves #2965
Resolves #2877

@iblancasa iblancasa requested a review from a team as a code owner April 18, 2024 14:44
@iblancasa
Copy link
Contributor Author

@IshwarKanse can you check if the behaviour is correct now? The test passed in my cluster but I need another couple of eyes.

@yuriolisa
Copy link
Contributor

@IshwarKanse can you check if the behaviour is correct now? The test passed in my cluster but I need another couple of eyes.

AFAIK, we fixed that here.

Copy link
Contributor

@jaronoff97 jaronoff97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO rather than creating one service monitor for the self scrape and the prom exporter (which I think are on different ports) we should just make two service monitors. I worry that this is going to break users because it will cause dual scraping

internal/manifests/collector/service.go Show resolved Hide resolved
internal/manifests/collector/servicemonitor.go Outdated Show resolved Hide resolved
@iblancasa
Copy link
Contributor Author

@IshwarKanse can you check if the behaviour is correct now? The test passed in my cluster but I need another couple of eyes.

AFAIK, we fixed that here.

The SM has the label for the monitor Service but not for the collector. So, it doesn't match the collector and the Prometheus exporter is never scrapped.

…sent

Signed-off-by: Israel Blancas <iblancasa@gmail.com>
@pavolloffay
Copy link
Member

it seems like e2e tests are failing

@IshwarKanse
Copy link
Contributor

The PR needs to be updated.

Signed-off-by: Israel Blancas <iblancasa@gmail.com>
Signed-off-by: Israel Blancas <iblancasa@gmail.com>
Signed-off-by: Israel Blancas <iblancasa@gmail.com>
@IshwarKanse
Copy link
Contributor

@iblancasa We need to update this test as well e2e-openshift/otlp-metrics-traces

Copy link
Contributor

@jaronoff97 jaronoff97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approach broadly makes sense, a few thoughts/questions.

internal/manifests/collector/servicemonitor.go Outdated Show resolved Hide resolved
internal/manifests/collector/service.go Outdated Show resolved Hide resolved
Signed-off-by: Israel Blancas <iblancasa@gmail.com>
.chloggen/bug_2877.yaml Outdated Show resolved Hide resolved
"params.OtelCol.namespace", params.OtelCol.Namespace,
)

if !params.OtelCol.Spec.Observability.Metrics.EnableMetrics {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

personal opinion but this statement can be written in a more readable way

if params.OtelCol.Spec.Observability.Metrics.EnableMetrics &&  params.Config.PrometheusCRAvailability() == prometheus.NotAvailable && params.OtelCol.Spec.Mode == v1beta1.ModeSidecar{
  return true
}

l..V(2).Info("PodMonitor will not be created", "Metrics.enabledMetrics", "params.OtelCol.Spec.Observability.Metrics.EnableMetrics ", ....)
return false

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and I would move the logger creation close to returning false

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why a double condition + no log message is easier to read than the current approach?

The current is easy to follow and understand why the method returns false.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A single log message with all parameters makes it clear to understand the overall state.

The function has 2 states (true and false), aligning the condition to it makes it easier to understand

@@ -78,6 +85,25 @@ func ServiceMonitor(params manifests.Params) (*monitoringv1.ServiceMonitor, erro
return &sm, nil
}

func shouldCreateServiceMonitor(params manifests.Params) bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

personal opinion but this statement can be written in a more readable way

See the previous comment

Copy link
Contributor

@swiatekm-sumo swiatekm-sumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, minor nitpick about the changelog.

.chloggen/bug_2877.yaml Outdated Show resolved Hide resolved
Signed-off-by: Israel Blancas <iblancasa@gmail.com>
@iblancasa iblancasa requested a review from jaronoff97 May 13, 2024 13:57
@pavolloffay
Copy link
Member

@jaronoff97 could you please re-review?

Copy link
Contributor

@jaronoff97 jaronoff97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one thought to prevent this from breaking users

const (
headlessLabel = "operator.opentelemetry.io/collector-headless-service"
monitoringLabel = "operator.opentelemetry.io/collector-monitoring-service"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iblancasa i think we should keep these around for now, mark them as deprecated and remove them in two+ versions rather than deleting them outright which would constitute a breaking change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Signed-off-by: Israel Blancas <iblancasa@gmail.com>
@iblancasa iblancasa requested a review from jaronoff97 May 17, 2024 16:05
Signed-off-by: Israel Blancas <iblancasa@gmail.com>
@jaronoff97 jaronoff97 enabled auto-merge (squash) May 17, 2024 16:51
@iblancasa
Copy link
Contributor Author

I think it should be fine now. I'm having some problems running this locally due to some network restrictions...

@jaronoff97
Copy link
Contributor

Signed-off-by: Israel Blancas <iblancasa@gmail.com>
auto-merge was automatically disabled May 19, 2024 02:35

Head branch was pushed to by a user without write access

@iblancasa
Copy link
Contributor Author

@jaronoff97 thanks! I fixed the issue now. I added the label in an extra service.

@Snoopy666
Copy link

Description:: fix the labels for the Service Monitor integration. Exclude the headless Service from the integration with SM.

Resolves #2965
Resolves #2877

@pavolloffay pavolloffay merged commit 13bc62d into open-telemetry:main May 20, 2024
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants