Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LOG-2811: Add lokistack output type to cluster forwarder #2483

Merged
merged 1 commit into from
May 14, 2024

Conversation

Clee2691
Copy link
Contributor

Description

  • Refactored Loki to use observability API
  • Added Loki output to API
  • Migrates LokiStack output to Loki outputs crafting the URL with the tenant (current behavior of LokiStack)

/cc
/assign

Links

@openshift-ci-robot
Copy link

openshift-ci-robot commented May 10, 2024

@Clee2691: This pull request references LOG-2811 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.8.0" version, but no target version was set.

In response to this:

Description

  • Refactored Loki to use observability API
  • Added Loki output to API
  • Migrates LokiStack output to Loki outputs crafting the URL with the tenant (current behavior of LokiStack)

/cc
/assign

Links

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 10, 2024
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 10, 2024
Copy link
Contributor

openshift-ci bot commented May 10, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@Clee2691
Copy link
Contributor Author

/test all

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 10, 2024
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 10, 2024
Copy link
Contributor

@alanconway alanconway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from a comment on the auth debate, this looks good.
Generating extra outputs and pipelines is somewhat complex but I think it is the only option right now - vector doesn't support any way to vary the base URL per message.
I'd like the code to be simpler, but I don't have any good idea how to do that right now....

api/observability/v1/clusterlogforwarder_types.go Outdated Show resolved Hide resolved
@Clee2691 Clee2691 marked this pull request as ready for review May 10, 2024 17:51
@Clee2691 Clee2691 changed the title [WIP] LOG-2811: Add lokistack output type to cluster forwarder LOG-2811: Add lokistack output type to cluster forwarder May 10, 2024
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 10, 2024
@openshift-ci openshift-ci bot requested review from cahartma and jcantrill May 10, 2024 17:52
@Clee2691
Copy link
Contributor Author

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 10, 2024
api/observability/v1/cluster_log_forwarder.go Outdated Show resolved Hide resolved
api/observability/v1/cluster_log_forwarder.go Outdated Show resolved Hide resolved
api/observability/v1/cluster_log_forwarder.go Outdated Show resolved Hide resolved
api/observability/v1/output_types.go Outdated Show resolved Hide resolved
internal/generator/vector/helpers/secrets.go Outdated Show resolved Hide resolved
internal/generator/vector/output/loki/loki.go Outdated Show resolved Hide resolved
internal/generator/vector/output/loki/loki_conf_test.go Outdated Show resolved Hide resolved
internal/logstore/observability/lokistack/lokistack.go Outdated Show resolved Hide resolved
test/observability/helpers/generator.go Outdated Show resolved Hide resolved
@Clee2691
Copy link
Contributor Author

/retest

Copy link
Contributor

@jcantrill jcantrill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 small nits othewise lgtm

// to allow dynamic per-event values. Defaults to the log type (i.e. application, audit, infrastructure)
//
// +kubebuilder:validation:Required
// +kubebuilder:default:={{.log_type}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we discovered this needs quote? Is that right?

}

// hasLokiStackOutput checks if a pipeline includes a lokistack output
func hasLokiStackOutput(outputMap map[string]*obs.OutputSpec, outputRefs []string) *sets.String {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit. since we need to change the other thing maybe make this plural?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the name of this function to better represent what it does: findLokiStackOutputs()

@jcantrill
Copy link
Contributor

/approve

Copy link
Contributor

openshift-ci bot commented May 14, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Clee2691, jcantrill

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 14, 2024
@Clee2691
Copy link
Contributor Author

/hold cancel

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 14, 2024
Copy link
Contributor

@jcantrill jcantrill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label May 14, 2024
Copy link
Contributor

openshift-ci bot commented May 14, 2024

@Clee2691: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit f9c30d9 into openshift:master May 14, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. release/6.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants