Trace with Huge Amount of Spans

[QHQuach]

One of our services creates a trace that has roughly 10K of spans. The trace is not shown properly in the Uptrace Community Edition UI. Is there a limit to number of spans within a given trace before the UI chokes? Or maybe there is a configuration some where to have UI perform lazy loading (e.g., don't load the children until the parent is expanded, and yes, there might not be avail metric of the whole trace until all spans are loaded)?

Also, would Enterprise Edition handle/support such traces with large amount of spans within based on this statement (https://uptrace.dev/get/enterprise.html#huge-traces)

Thanks in advance.

[vmihailenco]

Yes, Uptrace EE is able to display large traces with 100k spans and more. You can try Uptrace Cloud to see how we handle it.

Is there a limit to number of spans within a given trace before the UI chokes?

We currently limit the trace size to 10k spans, but it looks like even that is too much. Do you suggest to make this limit configurable?

Or maybe there is a configuration some where to have UI perform lazy loading (e.g., don't load the children until the parent is expanded, and yes, there might not be avail metric of the whole trace until all spans are loaded)?

The UI will be slow and there is no telling when it will break, e.g. expanding the span may load the same 10k spans.

[QHQuach]

Do you suggest to make this limit configurable?

Not so much suggesting a limit configuration but more of a lazy-fetching design/area for the front-end for certain traces that have large number of spans. For example, the children spans are not fetched for display until a parent span is expanded. Of courses with this approach, the trace won't have the total calculated timeline/duration value since all spans are not fetched and returned at once to the front-end. This limitation surely can be accepted/understandable for these kinds of trace. All other normal traces will still have the normal design (fetch all for display and value calculation).

We currently limit the trace size to 10k spans, but it looks like even that is too much.

I am working on reducing spans wherever I can. However, this specific service of ours is processing 2500+ records in a batch. Each record processing has a lot of things going on and I am trying to provide the info to the support/dev/customer. With your confirmation above, I will now try to see if I can break the telemetry up into multiple traces instead of a single one.

BTW, thank you for your quick reply on the matter. I also see that you have created an enhancement request for 10K+ spans notif. That would help too.