-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[out_loki] A lot of warning 'Tenant ID is overwritten A -> B' if tenant_id_key is used #8809
Comments
Why you used fluentbit not promtail to send data to Loki, I think promtail will good to use when with Loki |
@YevhenLodovyi, are your log entries separated into different chunks, e. g. by making sure that they get re-emitted with a tag that contains the actual value of the tenant_id? Have a look at #2935 (comment), that describes the issue quite well. It would be helpful if you provided the lua script and the rest of the fluent-bit config. If your chunks are well aligned: We are possibly encountering the same issue, where a race condition seems to be involved. I guess it was introduced (or at least not fixed) by PR #6931 where But I have to admit that I don't see a context switch in between. Maybe @leonardo-albertovich could shed some light on it? Though we are quite sure that the value changes in between, as we captured the traffic using tcpdump and the chunks are well aligned and contain log messages from (in our case) distinct namespaces only, but are tagged with the wrong X-Scope-OrgID header. We have configured
If this is a separate topic, I'll file an additional issue. Update: We recompiled fluent-bit with |
Hello,
Your Environment
I am using flb to send logs to loki. I am trying to seperate logs, so I am using multi-tenancy. I have a lua script to generate the tenant_id, so in the output i have:
As far as I can see the logs are distributed properly, but I have a lot of warning:
The warn is defined here: https://github.com/fluent/fluent-bit/blob/master/plugins/out_loki/loki.c#L1152
The text was updated successfully, but these errors were encountered: