New component: sumologicprocessor

[sumo-drosiek]

The purpose and use-cases of the new component

This is vendor specific processor. Schema used by Sumo Logic differs from OpenTelemetry ones, and because of that we need some translations and additional data pre-processing before sending data to Sumo Logic.

More details are available in the component README

Example configuration for the component

processors:
  sumologic:
    # Defines whether the `cloud.namespace` resource attribute should be added.
    # default = true
    add_cloud_namespace: {true,false}

    # Defines whether attributes should be translated
    # from OpenTelemetry to Sumo Logic conventions;
    # see "Attribute translation" documentation chapter from this document.
    # default = true
    translate_attributes: {true,false}

    # Specifies whether telegraf metric names should be translated to match
    # Sumo Logic conventions expected in Sumo Logic host related apps (for example
    # `procstat_num_threads` => `Proc_Threads` or `cpu_usage_irq` => `CPU_Irq`).
    # See `translate_telegraf_metrics_processor.go` for full list of translations.
    # default = true
    translate_telegraf_attributes: {true, false}

    # Specifies whether docker stats metric names should be translated to match
    # Sumo Logic conventions expected in Sumo Logic host related apps (for example
    # `container.cpu.usage.kernelmode` => `cpu_usage.usage_in_kernelmode` or `container.memory.usage.max` => `max_usage`).
    # It also translates some resource attribute names (for example `container.id` => `container.FullID`).
    # See `translate_docker_metrics_processor.go` for full list of translations.
    # default = false
    translate_docker_metrics: {true, false}

    # Specifies if attributes should be nested, basing on their keys.
    # See "Nesting attributes" documentation chapter from this document.
    nest_attributes:
      # Defines whether attributes should be nested.
      # default = false
      enabled: {true, false}

      # Defines the string used to separate key names in attributes that are to be nested.
      # default = "."
      separator: <separator>

      # Defines a list of allowed prefixes to be nested.
      # For example, if "kubernetes." is in this list, then all "kubernetes.*" attributes will be nested.
      # default = []
      include: [<prefix>]

      # Defines a list of prefixes not allowed to be nested.
      # For example, if "k8s." is in this list, then all "k8s.*" attributes will not be nested.
      # default = []
      exclude: [<prefix>]

      # If enabled, then maps that would have only one value will be squashed.
      # For example,{"k8s": {"pods": {"a": "A", "b": "B"}}}
      # will be squashed to {"k8s.pods": {"a": "A", "b": "B"}}
      # default = false
      squash_single_values: {true, false}

    # Specifies if attributes matching given pattern should be mapped to a common key.
    # See "Aggregating attributes" documentation chapter from this document.
    # default = []
    aggregate_attributes:
      - attribute: <attribute>
        prefixes: [<prefix>]
      - ...

    # Defines whether attribute should be added to record attributes.
    # Currently available attributes are `severity_number`, `severity_text`,
    # `trace_id`, `span_id`
    # all are disabled by default, enabled: false
    field_attributes:
      severity_number:
        enabled: true
        name: "loglevel"

Telemetry data types supported

logs, metrics, traces

Is this a vendor-specific component?

• This is a vendor-specific component
• If this is a vendor-specific component, I am proposing to contribute this as a representative of the vendor.

Sponsor (optional)

No response

Additional context

This is required in order to move other Sumo Logic specific components from Sumo Logic repository to this one.

[Frapschen]

Schema used by Sumo Logic differs from OpenTelemetry ones, and because of that we need some translations and additional data pre-processing before sending data to Sumo Logic.

As that said, it's more likely an exporter, why not add those functions into the sumologicexporter?

[sumo-drosiek]

As that said, it's more likely an exporter, why not add those functions into the sumologicexporter?

We don't think that exporter should modify data and also with this processor, OTLP exporter may be use to send data to Sumo Logic. We also believe that it is important to be able to send same data to different targets, e.g. local databases, s3 and so on.

Finally this processor is not only about schema, but also has some features our customer asks as for and which are not supported by other processors yet.

[andrzej-stencel]

It' worth noting that this processor is currently maintained in the Sumo Logic distro repo as Sumo Logic Schema processor.

We want to move this processor to be maintained in the contrib repo together with the already existing Sumo Logic exporter and also Sumo Logic extension coming soon (also moved from the Sumo distro repo).

[andrzej-stencel]

I'm willing to sponsor this component.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[andrzej-stencel]

This is not stale, work is ongoing in #23949.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[andrzej-stencel]

The Sumo Logic processor is now merged in contrib repository, but we need to include it in the contrib release.

[sumo-drosiek]

The Sumo Logic processor is now merged in contrib repository, but we need to include it in the contrib release.

#29509 should resolve the issue