Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option to export both past and future events #4203

Merged
merged 5 commits into from
May 31, 2024

Conversation

IyeOnline
Copy link
Contributor

@IyeOnline IyeOnline commented May 10, 2024

  • The export operators now features a --retro
    flag. This flag will make the operators export past events, even when --live
    is set:

    • Specifying both --retro --live will first export all past events and
      then go into live mode.
    • Specifying only --live behaves like it did previously. It only exports
      new events.
    • Specifying only --retro behaves as if the flag was not set, just like
      previously without any flag.
    • Specifying none of the flags is also unchanged.

    The combination of both flags is experimental. It has an obvious issue with events that happen live while exporting previous events.

  • The diagnostics and metrics operators also feature these flags, with the
    same behavior.

  • A one batch delay in export --live has been fixed.

@IyeOnline IyeOnline changed the title Add an option to get export both past and future events Add an option export both past and future events May 10, 2024
@IyeOnline IyeOnline added feature New functionality bug Incorrect behavior operator Source, transformation, and sink labels May 10, 2024
@IyeOnline IyeOnline changed the title Add an option export both past and future events Add an option to export both past and future events May 10, 2024
Copy link
Member

@dominiklohmann dominiklohmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can and should be merged—even with the gap in the data that we currently have between the two modes this still adds quite some value.

@dominiklohmann dominiklohmann merged commit 71b0190 into main May 31, 2024
21 of 22 checks passed
@dominiklohmann dominiklohmann deleted the topic/unified-export-operator branch May 31, 2024 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior feature New functionality operator Source, transformation, and sink
Projects
None yet
2 participants