[Feature]: Support Context propagation for jaeger consumer

[SwanHtetAung]

Requirement

Use header value to create a new context instead of creating a plain background context.

Problem

Currently, when jaeger consumers consume messages from Sarma, it does not use the header provided by kafka message. It instead creates a new context.

Proposal

I believe the ingester should examine the context to appropriately configure tracing for the Jaeger system. Currently, the collector sends a message to Kafka, and the ingester consumes that message. However, because the ingester doesn't read from the context, it invariably initiates a new root span.

If tracing is activated for both the collector and the ingester, two separate traces are currently produced: one for the collector and another for the ingester.

By enabling the ingester to read from the context, It would consolidate these into a single trace. In short, it will show a path from collector -> ingester

Open questions

No response

[SwanHtetAung]

https://github.com/jaegertracing/jaeger/blob/main/cmd/ingester/app/processor/span_processor.go#L70

[yurishkuro]

there is nothing that ingester needs to read from the context. Please explain your use case.

[Adarsh-jaiss]

• Hey @SwanHtetAung , I would love to work on this issue, can you please elaborate more about the issue, or any reference you can provide!

[SwanHtetAung]

there is nothing that ingester needs to read from the context. Please explain your use case

I believe the ingester should examine the context to appropriately configure tracing for the Jaeger system. Currently, the collector sends a message to Kafka, and the ingester consumes that message. However, because the ingester doesn't read from the context, it invariably initiates a new root span.

If tracing is activated for both the collector and the ingester, two separate traces are currently produced: one for the collector and another for the ingester.

By enabling the ingester to read from the context, It would consolidate these into a single trace. In short, it will show a path from collector -> ingester.

[yurishkuro]

So your use case is tracing the flow through Jaeger itself - that's fair (if a bit dangerous / recursive).

[SwanHtetAung]

what do you mean by a bit dangerous / recursive here?

[yurishkuro]

You are tracing the system that captures traces - if not careful you can easily create an infinite loop. In Jaeger we historically avoided tracing of the ingestion path, only query paths are traced.

[SwanHtetAung]

I see. Do you have example cases an infinite loop might happen? I was thinking this away.
Agent -> Collector -> Ingester -> Storage trace path should exists and use the same context(Ideally from agent). How can an infinite loop happen there?

[yurishkuro]

Your applications start a trace A and report it to Jaeger. Jaeger receives the trace data, and in the pipeline you described above starts another trace B. Trace B is also reported as separate payload to Jaeger, which may start trace C for processing that payload, and so on.

[SwanHtetAung]

I see, did not think of that case.

[yurishkuro]

I am going to close this - we do not have plans to support tracing of the ingestion path.