New component: sumologicprocessor #23949

sumo-drosiek · 2023-07-04T10:17:50Z

Description:

Adds Sumo Logic Processor

The Sumo Logic processor (config name: sumologic)
modifies the metadata on logs, metrics and traces sent to Sumo Logic
so that the Sumo Logic apps can make full use of the ingested data.

Link to tracking Issue: #23946

Testing:

Unit tests
Component is being used by our customers

Documentation: README.md

sumo-drosiek · 2023-07-04T10:19:12Z

I may try to split that PR to smaller ones if it'll help. Now it is just moved from Sumo Logic repository and adjusted due to Adding nee components instruction

sumo-drosiek · 2023-07-04T11:15:17Z

cc: @astencel-sumo @aboguszewski-sumo

aboguszewski-sumo · 2023-07-04T13:56:54Z

processor/sumologicprocessor/README.md

+    nest_attributes:
+      # Defines whether attributes should be nested.
+      # default = false
+      enabled: {true, false}
+
+      # Defines the string used to separate key names in attributes that are to be nested.
+      # default = "."
+      separator: <separator>
+
+      # Defines a list of allowed prefixes to be nested.
+      # For example, if "kubernetes." is in this list, then all "kubernetes.*" attributes will be nested.
+      # default = []
+      include: [<prefix>]
+
+      # Defines a list of prefixes not allowed to be nested.
+      # For example, if "k8s." is in this list, then all "k8s.*" attributes will not be nested.
+      # default = []
+      exclude: [<prefix>]
+
+      # If enabled, then maps that would have only one value will be squashed.
+      # For example,{"k8s": {"pods": {"a": "A", "b": "B"}}}
+      # will be squashed to {"k8s.pods": {"a": "A", "b": "B"}}
+      # default = false
+      squash_single_values: {true, false}
+
+    # Specifies if attributes matching given pattern should be mapped to a common key.
+    # See "Aggregating attributes" documentation chapter from this document.
+    # default = []
+    aggregate_attributes:
+      - attribute: <attribute>
+        prefixes: [<prefix>]
+      - ...
+


For the record: these two features were created as a temporary solution for these issues: #14977 #14946

What would be the migration path once these are resolved?

Mark configuration as deprecated and add warning in case of using it and also instruction how to use new configuration

Remove support for the functionality, but leaving logs in order to avoid confusion with default missing configuration error

Remove logs and configuration option completely

aboguszewski-sumo · 2023-07-04T13:58:26Z

I don't know if it's needed, but I confirm my will to be a codeowner for this component.

processor/sumologicprocessor/metadata.yaml

github-actions · 2023-07-30T05:18:56Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

sumo-drosiek · 2023-07-31T06:24:13Z

@evan-bradley Could you take a look?

evan-bradley

@sumo-drosiek This PR is fairly large, so I only did a surface pass. I will try to go back and do an in-depth review within a week.

.github/CODEOWNERS

processor/sumologicprocessor/metadata.yaml

processor/sumologicprocessor/README.md

processor/sumologicprocessor/processor.go

github-actions · 2023-08-19T05:20:15Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

sumo-drosiek · 2023-08-21T05:32:08Z

@evan-bradley

@sumo-drosiek This PR is fairly large, so I only did a surface pass

Thank you, I made requested changes

github-actions · 2023-09-05T05:20:24Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

processor/sumologicprocessor/README.md

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

Co-authored-by: Daniel Jaglowski <jaglows3@gmail.com>

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

Co-authored-by: Andrzej Stencel <astencel@sumologic.com>

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

…gation Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

**Description:** Adds Sumo Logic Processor The Sumo Logic processor (config name: `sumologic`) modifies the metadata on logs, metrics and traces sent to [Sumo Logic][sumologic_webpage] so that the Sumo Logic [apps][sumologic_apps] can make full use of the ingested data. [sumologic_webpage]: https://www.sumologic.com [sumologic_apps]: https://www.sumologic.com/applications/ **Link to tracking Issue:** open-telemetry#23946 **Testing:** - Unit tests - Component is being used by our customers **Documentation:** `README.md` --------- Signed-off-by: Dominik Rosiek <drosiek@sumologic.com> Co-authored-by: Daniel Jaglowski <jaglows3@gmail.com> Co-authored-by: Andrzej Stencel <astencel@sumologic.com>

sumo-drosiek requested a review from a team as a code owner July 4, 2023 10:17

sumo-drosiek requested a review from jpkrohling July 4, 2023 10:17

github-actions bot assigned evan-bradley Jul 4, 2023

github-actions bot added the cmd/otelcontribcol otelcontribcol command label Jul 4, 2023

sumo-drosiek changed the title ~~New component: sumologicschemaprocessor~~ New component: sumologicprocessor Jul 4, 2023

sumo-drosiek force-pushed the drosiek-sumologic-schema-processor branch from 29a7a55 to 72356b3 Compare July 4, 2023 11:12

sumo-drosiek force-pushed the drosiek-sumologic-schema-processor branch from 7ed6d9f to 02205a2 Compare July 4, 2023 13:31

aboguszewski-sumo reviewed Jul 4, 2023

View reviewed changes

sumo-drosiek force-pushed the drosiek-sumologic-schema-processor branch 4 times, most recently from 9d4e8f6 to de15738 Compare July 5, 2023 10:25

djaglowski reviewed Jul 5, 2023

View reviewed changes

processor/sumologicprocessor/metadata.yaml Outdated Show resolved Hide resolved

atoulme added the Accepted Component New component has been sponsored label Jul 16, 2023

github-actions bot added the Stale label Jul 30, 2023

github-actions bot removed the Stale label Aug 1, 2023

evan-bradley reviewed Aug 2, 2023

View reviewed changes

github-actions bot added the Stale label Aug 19, 2023

github-actions bot removed the Stale label Aug 22, 2023

github-actions bot added the Stale label Sep 5, 2023

andrzej-stencel reviewed Sep 5, 2023

View reviewed changes

processor/sumologicprocessor/README.md Outdated Show resolved Hide resolved

andrzej-stencel reviewed Sep 5, 2023

View reviewed changes

processor/sumologicprocessor/README.md Outdated Show resolved Hide resolved

sumo-drosiek and others added 22 commits October 30, 2023 13:36

chore: rename sumologicschemaprocessor to sumologicprocessor

76c2a1e

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore: add metadaa.yaml and add component to the builder

bd89ad8

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore: add codeowners

cad2258

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

refactor: refactor due to lint check

3dc1953

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

Update processor/sumologicprocessor/metadata.yaml

e74ab2a

Co-authored-by: Daniel Jaglowski <jaglows3@gmail.com>

chore: add @astencel-sumo as codeowner

de6ef63

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore: remove contrib from supported distributions

948601a

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore: re-word some messages due to review

65a6ca4

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

Update processor/sumologicprocessor/README.md

09c25f9

Co-authored-by: Andrzej Stencel <astencel@sumologic.com>

Update processor/sumologicprocessor/README.md

9d4bf9b

Co-authored-by: Andrzej Stencel <astencel@sumologic.com>

Update .chloggen/drosiek-sumologic-schema-processor.yaml

53fad59

Co-authored-by: Andrzej Stencel <astencel@sumologic.com>

feat(sumologicprocessor): check for conflicts in prefix names

da57af8

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore(sumologicprocessor): fix go.mod

f565edd

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore(sumologicprocessor): fix according to apicheck

0b0f5d8

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore: go mod tidy

500ab1e

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

feat(sumologicprocessor): check for duplication of attribute names

5cf503b

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore(sumologicprocessor): restore original code for attributes aggre…

25324f0

…gation Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore: fix versions in go.mod

81a0bee

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore: update accoreding to review

aeaade5

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore: refactor due to review

ec4ef40

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore: refactor due to review

6e169ef

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore: change semcov path

e819f3b

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

sumo-drosiek force-pushed the drosiek-sumologic-schema-processor branch from 15de60c to e819f3b Compare October 30, 2023 12:46

sumo-drosiek added 2 commits October 30, 2023 15:45

chore: fix checks

5435670

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

chore: gendependabot

909973a

Signed-off-by: Dominik Rosiek <drosiek@sumologic.com>

dmitryax approved these changes Oct 30, 2023

View reviewed changes

dmitryax merged commit e0c15a4 into open-telemetry:main Oct 30, 2023
82 of 83 checks passed

github-actions bot added this to the next release milestone Oct 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New component: sumologicprocessor #23949

New component: sumologicprocessor #23949

sumo-drosiek commented Jul 4, 2023 •

edited

sumo-drosiek commented Jul 4, 2023 •

edited

sumo-drosiek commented Jul 4, 2023 •

edited

aboguszewski-sumo Jul 4, 2023

MovieStoreGuy Oct 9, 2023

sumo-drosiek Oct 10, 2023

aboguszewski-sumo commented Jul 4, 2023

github-actions bot commented Jul 30, 2023

sumo-drosiek commented Jul 31, 2023

evan-bradley left a comment

github-actions bot commented Aug 19, 2023

sumo-drosiek commented Aug 21, 2023

github-actions bot commented Sep 5, 2023

New component: sumologicprocessor #23949

New component: sumologicprocessor #23949

Conversation

sumo-drosiek commented Jul 4, 2023 • edited

sumo-drosiek commented Jul 4, 2023 • edited

sumo-drosiek commented Jul 4, 2023 • edited

aboguszewski-sumo Jul 4, 2023

Choose a reason for hiding this comment

MovieStoreGuy Oct 9, 2023

Choose a reason for hiding this comment

sumo-drosiek Oct 10, 2023

Choose a reason for hiding this comment

aboguszewski-sumo commented Jul 4, 2023

github-actions bot commented Jul 30, 2023

sumo-drosiek commented Jul 31, 2023

evan-bradley left a comment

Choose a reason for hiding this comment

github-actions bot commented Aug 19, 2023

sumo-drosiek commented Aug 21, 2023

github-actions bot commented Sep 5, 2023

sumo-drosiek commented Jul 4, 2023 •

edited

sumo-drosiek commented Jul 4, 2023 •

edited

sumo-drosiek commented Jul 4, 2023 •

edited