External ServiceMonitor deleted by controller

[phillebaba]

Describe the bug

Then controller will assume that any ServiceMonitor resource which has a matching name to the Thanos resource is owned by the controller. This causes a ServiceMonitor with the name foo-compactor to be deleted if there is a Thanos resource in the same namespace with the name foo.

Steps to reproduce the issue:

Create a Thanos resource with the name foo whit a compactor configuration. The important thing is that it does not have any monitor configuration for the compactor. Create a ServiceMonitor in the same namespace with the name foo-compactor. Wait until the controller reconciles the Thanos resource and deletes the ServiceMonitor.

Expected behavior

The service monitor created outside of the controller should never be modified or deleted by the controller as it is not the owner of the resource.

Screenshots

N/A

Additional context

I think the reason this is occurring is because of this logic. It basically assumes that if there is no service monitor configuration it should delete any service monitor with a given name, even if the operator has not created the service monitor.

thanos-operator/pkg/resources/compactor/servicemonitor.go

Lines 53 to 56 in 1bd4c3c

	delete := &prometheus.ServiceMonitor{
	ObjectMeta: c.getMeta(),
	}
	return delete, reconciler.StateAbsent, nil

[michaelajr]

I have seen this behavior as well. It happens with any object you create on your own that matches a name the operator thinks it manages. IMO the operator should look at a managed-by label as well as the name.

[pepov]

Created an issue to make this possible using the returned desiredstate:
https://github.com/banzaicloud/operator-tools/issues/72