[Feature Request] Deduplication of attributes

[cijothomas]

As of today, Logs API/SDK does not do de-deuplication.
Tracing API/SDK does do de-duplication for SpanAttributes, but it is being removed. There is no de-duplication done for Event Attributes and Link Attributes.
Metrics API/SDK does do de-duplication, but it could be removed.

Please comment on this issue, if you are an end-user who want a feature from this repo to do the de-duplication. This is typically the case, if you, as instrumentation author, cannot enforce it, and your backend does not support it either. When you comment here, please describe the scenario in detail, so that we can design a solution to cover that. In general, if we chose to offer a feature for this, it'll be opt-in basis, as there is significant overhead to do de-duplication.

[hdost]

I'd like to bring up #1297

[hdost]

I'd like to bring up #1297

As discussed this is not necessarily a deviation from the spec, but I still think this might be worthy of some sort of mention that by default we don't ship with de-duplication. However, this is interesting as I believe it only applies to attributes and not necessarily Resources.

[hdost]

Discussion at the SIG yesterday made me think that we do want to be consistent with our API around around this, so we may want to have a followup to #1293 which behaves in a way similar to our AttributeSet discussion.

[cijothomas]

open-telemetry/opentelemetry-specification#3931 (comment) From spec, only applicable for logs, still in discussion only.