Manage Dashboards, Alerts and Functions with CRDs

[nico151999]

Hello everyone,

I got to try Openobserve lately and find it pretty cool. However, I try to build my stuff reproducibly. This is possible up to the point of installation. However, I couldn't find an existing solution that allows me to configure Dashboards, Alerts or Functions in a GitOps manner. Either one builds a pipeline that persists definitions of these resource types via the REST-API or one applies a Kubernetes approach and creates CRDs that define those resources. A respective Operator would need to take care of applying these configurations then. I assume one could also solve it using Crossplane if one writes a Crossplane provider. Implementing such solutions should be possible since the required RPCs are defined in the API spec.

Now, I wonder if there is something out there already which does this job. If there is not, I would like to hear your thoughts about a feature like this.

[prabhatsharma]

There is no IaC capability present for OpenObserve today. Of course, you could do it manually by building a pipeline and making REST calls.

We have thought of 2 alternatives:

1. Have an operator that will accept IaC requests and apply them (As mentioned by you)
2. Direct git integration for dashboards, alerts, functions, and more where syncing can happen, like in a gitops model managed by OpenObserve (without external tools like argo/flux).

We plan to follow the 2nd approach. #2340

Open to suggestions.

[nico151999]

Thanks for the response. Your 2nd approach is a cool idea, too. Especially for those who don't use external tools to sync their manifests from Git to their cluster or those who do not want to manage their configuration using K8s. However, those who actually do will face a scenario they won't like: the connection details will be required to be managed by two different tools (i.e. the external GitOps tool in use and Openobserve itself). Therefore, I think it would make sense to give the user the option to decide. I don't think that one solution excludes the other. Implementing the feature you described in #2340 still allows for a controller to check for custom resource changes and make respective REST calls to apply the changes to Openobserve. The two approaches would stand in conflict if something like identical resource identifiers are expected to be synced only.

I would personally prefer the CRD solution since I like managing my resources using K8s. It offers mechanisms to track the whole lifecycle from creation over updates to deletion. If Openobserve would decide to implement its own solution for syncing with Git this would require some logic that allows not only to be checked which resources exist in a Git repo but also which resources used to be tracked via Git but were deleted since the last synchronisation (since the manifests in Git might only be a subset of alerts, dashboards, functions etc.). Since the approach using CRDs seems to be the solution which requires less effort for now (maybe you have a different idea which would make approach 2 just as easy) I would even propose to prioritise approach 1.

However, by now I am just one new user of Openobserve and not a maintainer. I assume you have the bigger picture of what most users want to see. Maybe the majority of users does not use K8s features extensively and therefore would not make much use of respective CRDs.

Once again thanks for the response. I really appreciate the rather quick answer. Hopefully, a feature like this (no matter if approach 1, 2 or yet something else) will find its way into this amazing tool.

[prabhatsharma]

OpenObserve is used by thousands in production, which means we have users using It in all kinds of environments—k8s, binary, LXC, just docker, and docker compose. By virtue of having a large user base, each of these subsets is a large number on its own.

I agree that both solutions are not exclusive. Regardless, we have some time before we can get to it. We will keep this thread open for the community to chime in and provide their feedback, too.