How to filter traces by request header

[raagulrakesh]

Hi ,
I have enabled traces in tempo using opentelemetery. I want to filter traces by request header but header not showing in traces filter. I have passing request header by OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST.

tempo configuration

{
"overrides.yaml": "overrides:
defaults:
global:
max_bytes_per_trace: 10000000
",
"tempo.yaml": "multitenancy_enabled: false
usage_report:
reporting_enabled: true
compactor:
compaction:
block_retention: 720h
distributor:
receivers:
jaeger:
protocols:
grpc:
endpoint: 0.0.0.0:14250
thrift_binary:
endpoint: 0.0.0.0:6832
thrift_compact:
endpoint: 0.0.0.0:6831
thrift_http:
endpoint: 0.0.0.0:14268
opencensus: null
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
ingester:
{}
server:
http_listen_port: 3100
storage:
trace:
backend: local
local:
path: /var/tempo/traces
wal:
path: /var/tempo/wal
querier:
{}
query_frontend:
{}
overrides:
per_tenant_override_config: /conf/overrides.yaml
"
}

grafana tempo search

[joe-elliott]

If the header is an array type we currently do not have the ability to query it. The team is working on vparquet4 actively which will extend the parquet format to support queries on these data types.

The syntax for querying arrays is defined here:

https://github.com/grafana/tempo/blob/main/docs/design-proposals/2023-11%20TraceQL%20Extensions.md#arrays

After vp4 is merged we will be able to add these extensions.

[raagulrakesh]

I have tried querying array still I didn't get result for the query

[joe-elliott]

Arrays are not currently supported. The doc i linked to you shows our proposed syntax. We are actively working on merging vp4 and adding support for arrays.

cc @stoewer

[raagulrakesh]

Hi,
When can we expect this in live?

Thanks in advance.

[joe-elliott]

Hoping to tackle it in the next few months.

[raagulrakesh]

Is there any other options to filter temporally for now

[joe-elliott]

Currently you would need a different attribute to query. You could likely use the otel transform processor in either Grafana Alloy or the OTel Collector to write one automatically in your pipeline:

https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor