Added CGroupv2 support into Docker Extensions #839

anoopbabu29 · 2022-12-16T20:52:11Z

Which problem is this PR solving?

Adding a detector for the container id for cgroupv2 described in open-telemetry/opentelemetry-js-contrib#1173.

Short description of the changes

Generalized detector for docker v1, v2 and container id. Based on open-telemetry/opentelemetry-js-contrib#1181.

… for the value before hostname

…otnet-contrib into cgroupv2-sup

alexeypukhov · 2022-12-19T19:20:26Z

src/OpenTelemetry.Extensions.Docker/Resources/DockerResourceDetector.cs

+    {
+        try
+        {
+            return this.ExtractContainerIdV1(path) ?? this.ExtractContainerIdV2(path);


just thinking out loud.
If the cgroup path will match the Id V2 (e.g. "13:name=systemd:/pod/d86d75589bf6cc254f3e2cc29debdf85dde404998aa128997a819ff991827356/hostname") the only way ExtractContainerIdV1() won't return you "hostname" is (I assume, please correct me if I'm wrong) because of EncodingUtils.IsValidHexString() that won't treat this string valid.
So my thoughts:

it's a bit too advanced in the V1 logic to finally decide that this is the wrong string (I mean it goes thru a lot of code before realizing this)

it's a bit fragile and is very easy to make a bug because from a IsValidHexString() context it's not clear how important this code for this place here. I know it's covered by unit tests but still

So maybe since the V2 logic puts a very specific requirement on the format of the string we should run it first? It will early out much faster than V1

I agree with that logic. After reviewing it, I believe that the IsValidHexString() is the method that prevents the hostname from being accepted and switching the two would be helpful since, as you mentioned, it has a stricter requirement. One way to make IsValidHexString() less fragile is to perhaps make a more strict requirement like a character limit (64 characters)? But it is also unclear to me how crucial this method is so I definitely do not think that I should change it here.

…emetry-dotnet-contrib into cgroupv2-sup

utpilla · 2022-12-19T19:49:39Z

Adding the component owner for this project. @iskiselev Could you please review this PR?

iskiselev · 2022-12-19T20:27:54Z

src/OpenTelemetry.Extensions.Docker/Resources/DockerResourceDetector.cs

@@ -28,6 +28,8 @@ namespace OpenTelemetry.Extensions.Docker.Resources;
 public class DockerResourceDetector : IResourceDetector
 {
    private const string FILEPATH = "/proc/self/cgroup";
+    private const string FILEPATHV2 = "/proc/self/mountinfo";


I don't see any usage of FILEPATHV2 in detector. I'd guess, that ExtractContainerIdV2 should use it, instead of FILEPATH. Proper fix for it will also solve problem mentioned by @alexeypukhov , as that functions should check different file.

Change has been made to check both versions in the Detect method and assuming the version being used in the BuildResource method. This way, the problem mentioned by @alexeypukhov would be solved as you mentioned since no container id would be able to be found when checking for cgroupv1 if cgroupv2 is being used.

…ce assumes the version with default parameter

…emetry-dotnet-contrib into cgroupv2-sup

alexeypukhov · 2022-12-23T00:01:47Z

src/OpenTelemetry.Extensions.Docker/Resources/DockerResourceDetector.cs

    /// <returns>Returns Resource with list of key-value pairs of container resource attributes if container id exists else empty resource.</returns>
-    internal Resource BuildResource(string path)
+    internal Resource BuildResource(string path, bool isCGroupV1 = true)


you already have two sets of methods: one for V1 and another for V2. I'd keep the same for this method too. That way you don't have to carry isCGroupV1 parameter from method to method and you can call either BuildResourceV1() or BuildResourceV2() method in the Detect()

alexeypukhov · 2022-12-23T00:06:15Z

test/OpenTelemetry.Extensions.Docker.Tests/Resources/DockerResourceDetectorTests.cs

+        using (TempFile tempFile = new TempFile())
+        {
+            tempFile.Write(CGROUPLINEV2);
+            Assert.Equal(CONTAINERIDV2, this.GetContainerId(dockerResourceDetector.BuildResource(tempFile.FilePath, false)));


I'd use Detect() method for unit test instead of BuildResource(). Detect is part of the interface while BuildResource is just a part of internal implementation

Right now there is currently no reference to Detect in the tests, and I believe its due to considerations of writing to a specific file path on the machine. As such, the only aspect of the test that would be lost by not including this detect is the reading of the file at the designated filepath. I'm not sure if its a good idea to include Detect in the tests if this is the case.

Yes you are right. If we don't want to stick to the FILEPATH and FILEPATH2 locations you cannot use Detect() method. In that case I think you made the right choice to test BuildResourceV1 and BuildResourceV2 separately. At the end if you know which version you are writing to a temp file you should know which method to call to read from the file

…emetry-dotnet-contrib into cgroupv2-sup

alexeypukhov · 2022-12-23T01:18:39Z

test/OpenTelemetry.Extensions.Docker.Tests/Resources/DockerResourceDetectorTests.cs

@@ -57,6 +57,13 @@ public class DockerResourceDetectorTests
    private const string CONTAINERID =
        "d86d75589bf6cc254f3e2cc29debdf85dde404998aa128997a819ff991827356";

+    // cgroupv2 line with container Id
+    private const string CGROUPLINEV2 =


do you also need to check suffix or prefix for cgroup v2?
Also just a thought. Should we check prefix and suffix end to end, I mean with reading from file? If GetContainerId works with extracted cgroup string no matter V1 or V2, maybe we should have a separate test just for this function and do not mix it with reading the line from the file and parsing it depending on V1 or V2

I can check for them as they are separate implementations. I figured that most of the functionality was similar enough to warrant not having them but I believe you are correct that they should exist, although the suffix isn't a necessary test in my opinion since its not a real case I believe.

codecov · 2022-12-23T04:10:51Z

Codecov Report

Merging #839 (d043c33) into main (70f9f0f) will increase coverage by 0.00%.
The diff coverage is 80.00%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #839   +/-   ##
=======================================
  Coverage   79.07%   79.07%           
=======================================
  Files         182      182           
  Lines        5614     5629   +15     
=======================================
+ Hits         4439     4451   +12     
- Misses       1175     1178    +3

Impacted Files	Coverage Δ
...r.Stackdriver/Implementation/ActivityExtensions.cs	`75.00% <ø> (ø)`
...ons/Internal/ActivityEventAttachingLogProcessor.cs	`93.33% <0.00%> (ø)`
...on.EventCounters/MeterProviderBuilderExtensions.cs	`100.00% <ø> (ø)`
...elemetry.Instrumentation.Process/ProcessMetrics.cs	`100.00% <ø> (ø)`
...ensions.Docker/Resources/DockerResourceDetector.cs	`82.60% <77.77%> (-1.27%)`	⬇️
...ry.Extensions/Internal/DefaultLogStateConverter.cs	`87.50% <100.00%> (ø)`
...nsions/Logs/LogToActivityEventConversionOptions.cs	`100.00% <100.00%> (ø)`
...try.Extensions/Trace/AutoFlushActivityProcessor.cs	`81.48% <100.00%> (ø)`

github-actions · 2022-12-31T03:16:23Z

This PR was marked stale due to lack of activity. It will be closed in 7 days.

Kielek · 2023-01-02T06:15:36Z

@iskiselev based on the PR description package will support not only Docker, but also everything based on container id.
I suppose the package name should be changed from OpenTelemetry.Extensions.Docker to OpenTelemetry.Extensions.Container. Also all public API should be adjusted. I would keep it as separate PR.

BTW PR was marked as Stale, please let us know if you want to proceed with it.

pellared · 2023-01-02T15:03:59Z

I suggest to take a look at open-telemetry/opentelemetry-go#3508 for reference (especially the comments and code connected with testing)

iskiselev · 2023-01-05T00:49:53Z

test/OpenTelemetry.Extensions.Docker.Tests/Resources/DockerResourceDetectorTests.cs

+        using (TempFile tempFile = new TempFile())
+        {
+            tempFile.Write(INVALIDHEXCGROUPLINEV2);
+            Assert.Equal(dockerResourceDetector.BuildResourceV1(tempFile.FilePath), Resource.Empty);


Should it call BuildResourceV2, same as on line 202?

Or probably both. In that case we may need to add this tests to V1 also - as incorrect V2 lines should be still incorrect for V1.

Yes, that was my mistake, I'll make that change. And yes, I'll make a test checking that V1 fails for V2 and vice versa.

iskiselev · 2023-01-05T00:54:09Z

src/OpenTelemetry.Extensions.Docker/Resources/DockerResourceDetector.cs

+    /// </summary>
+    /// <param name="path">File path where container id exists.</param>
+    /// <returns>Returns Resource with list of key-value pairs of container resource attributes if container id exists else empty resource.</returns>
+    internal Resource BuildResourceV1(string path)


Source for BuildResourceV1 / BuildResourceV2 are very similar, same as ExtractContainerIdV1 / ExtractContainerIdV2.
To have less duplicated code, I'd probably create enum ParseMode {V1, V2} and pass it as second argument to BuildResource / ExtractContainerId with different logic called when you fetch string containerId = ... in ExtractContainerId.

iskiselev · 2023-01-05T01:01:20Z

test/OpenTelemetry.Extensions.Docker.Tests/Resources/DockerResourceDetectorTests.cs

+    private const string CONTAINERIDV2WITHPODMAN = "1a2de27e7157106568f7e081e42a8c14858c02bd9df30d6e352b298178b46809";
+
+    // Invalid cgroup line (contains a z)"
+    private const string INVALIDHEXCGROUPLINEV2 =


We have so many const defined in that file now. I like @pellared suggestion to look into structure of tests in open-telemetry/opentelemetry-go#3508 - they looks nice with a dictionary of named cased with input / expected result pairs.

I can also turn it into a theory with InlineData if that works better in this context

iskiselev · 2023-01-05T01:13:22Z

@iskiselev based on the PR description package will support not only Docker, but also everything based on container id. I suppose the package name should be changed from OpenTelemetry.Extensions.Docker to OpenTelemetry.Extensions.Container. Also all public API should be adjusted. I would keep it as separate PR.

@Kielek, the name can be changed, as more generic name OpenTelemetry.Extensions.Container - but we already have first release of it at https://www.nuget.org/packages/OpenTelemetry.Extensions.Docker - we will need to deprecate old package in NuGet and add link to new package name there. I don't know how to do it, but hope it won't be too hard. We can create separate issue and discuss it there. Looks like nodejs have already renamed it to container.

BTW PR was marked as Stale, please let us know if you want to proceed with it.
I've added review comments - was on holiday break before. Hope we should be able to merge it soon.

Kielek · 2023-01-05T05:43:11Z

@iskiselev based on the PR description package will support not only Docker, but also everything based on container id. I suppose the package name should be changed from OpenTelemetry.Extensions.Docker to OpenTelemetry.Extensions.Container. Also all public API should be adjusted. I would keep it as separate PR.

@Kielek, the name can be changed, as more generic name OpenTelemetry.Extensions.Container - but we already have first release of it at https://www.nuget.org/packages/OpenTelemetry.Extensions.Docker - we will need to deprecate old package in NuGet and add link to new package name there. I don't know how to do it, but hope it won't be too hard. We can create separate issue and discuss it there. Looks like nodejs have already renamed it to container.

Created #869. Most of the packages was previously released as OpenTelemetry.Contrib.Something.Something2, then renames to OpenTelemetry.Something.Something2. I think that @utpilla know how to handle such scenario.

iskiselev · 2023-01-06T21:08:22Z

test/OpenTelemetry.Extensions.Docker.Tests/Resources/DockerResourceDetectorTests.cs

+        {
+            Name = "Invalid hex cgroupv2 line (contains a z)",
+            Line = "13:name=systemd:/var/lib/containerd/io.containerd.grpc.v1.cri/sandboxes/fb5916a02feca96bdeecd8e062df9e5e51d6617c8214b5e1f3fz9320f4402ae6/hostname",
+            CgroupVersion = DockerResourceDetector.ParseMode.V1,


Isn't it test for DockerResourceDetector.ParseMode.V2?

Yes, let me fix that.

iskiselev

It looks good now.

iskiselev · 2023-01-09T17:15:18Z

@Kielek , @utpilla - the tests that failed looks not connected with this change:
OpenTelemetry.Instrumentation.EventCounters.Tests.EventCountersMetricsTests.EventCounterSameNameUsesNewestCreated
OpenTelemetry.Instrumentation.EventCounters.Tests.EventCountersMetricsTests.EventSourceNameShortening

Will somebody be able to restart tests (in case if it will pass on second run) or advice what can be done with them?

Kielek · 2023-01-09T17:17:09Z

Unfortunately, code coverego pipeline is unstable. Please do not bother if all Docker related tests passed. It is not flagged as mandatory, so @utpilla can merge after review.

* Changes Update CHANGELOG for 1.1.0-beta.1 release, that will include CGroup V2 support (open-telemetry#839, open-telemetry#693). It still have `-beta.1` suffix to make it prerelease until decision on package name will be done (open-telemetry#881, open-telemetry#869)

anoopbabu29 and others added 4 commits December 5, 2022 20:56

Initial changes

15b1b51

Adding cgroupv2 to test and updating the approach such that it checks…

4cb146d

… for the value before hostname

Merge branch 'main' of https://github.com/anoopbabu29/opentelemetry-d…

cb8f573

…otnet-contrib into cgroupv2-sup

Merge branch 'open-telemetry:main' into cgroupv2-sup

7ff1941

anoopbabu29 requested a review from a team as a code owner December 16, 2022 20:52

utpilla added the comp:resourcedetectors.container Things related to OpenTelemetry.ResourceDetectors.Container label Dec 19, 2022

alexeypukhov reviewed Dec 19, 2022

View reviewed changes

anoopbabu29 added 2 commits December 19, 2022 14:46

Merge branch 'cgroupv2-sup' of https://github.com/anoopbabu29/opentel…

aad5630

…emetry-dotnet-contrib into cgroupv2-sup

Respoding to Comments and build checks

94be6e9

iskiselev suggested changes Dec 19, 2022

View reviewed changes

anoopbabu29 and others added 4 commits December 19, 2022 15:30

Merge branch 'main' into cgroupv2-sup

856fed7

Adjusted build such that Detect checks both versions, and BuildResour…

fd0c8e4

…ce assumes the version with default parameter

Merge branch 'cgroupv2-sup' of https://github.com/anoopbabu29/opentel…

5ba7994

…emetry-dotnet-contrib into cgroupv2-sup

Merge branch 'main' into cgroupv2-sup

78a5a62

alexeypukhov reviewed Dec 23, 2022

View reviewed changes

anoopbabu29 added 2 commits December 22, 2022 20:01

Adjusting BuildResource at suggestion

14ce9b1

Merge branch 'cgroupv2-sup' of https://github.com/anoopbabu29/opentel…

492552d

…emetry-dotnet-contrib into cgroupv2-sup

alexeypukhov reviewed Dec 23, 2022

View reviewed changes

github-actions bot added the Stale label Dec 31, 2022

svrnm mentioned this pull request Jan 2, 2023

Add resource detectors to cart service open-telemetry/opentelemetry-demo#663

Merged

2 tasks

github-actions bot removed the Stale label Jan 3, 2023

anoopbabu29 and others added 2 commits January 4, 2023 16:56

Added same tests as the go version as well as their regex expression

a3647b3

Merge branch 'main' into cgroupv2-sup

9e05aba

iskiselev suggested changes Jan 5, 2023

View reviewed changes

Kielek mentioned this pull request Jan 5, 2023

Rename OpenTelemetry.Extensions.Docker to OpenTelemetry.ResourceDetectors.Container #869

Closed

Addressing Comments by adding Enums and cleaning up tests

b5857f0

github-actions bot assigned iskiselev Jan 5, 2023

Merge branch 'main' into cgroupv2-sup

7b7e580

iskiselev suggested changes Jan 6, 2023

View reviewed changes

Fix parse mode version in test

ba6b6ab

iskiselev approved these changes Jan 6, 2023

View reviewed changes

Kielek approved these changes Jan 9, 2023

View reviewed changes

Merge branch 'main' into cgroupv2-sup

d043c33

utpilla merged commit 7bd2fc9 into open-telemetry:main Jan 9, 2023

mmanciop pushed a commit to lumigo-io/opentelemetry-dotnet-contrib that referenced this pull request Jan 11, 2023

Added CGroupv2 support into Docker Extensions (open-telemetry#839)

e81267c

alanwest mentioned this pull request Jan 11, 2023

Rename OpenTelemetry.Extensions.Docker to OpenTelemetry.ResourceDetector.Container #881

Closed

2 tasks

iskiselev mentioned this pull request Jan 11, 2023

Add auto-detection of container information to resource if cgroupv2 is used #693

Closed

iskiselev mentioned this pull request Jan 12, 2023

[Extensions.Docker] Update CHANGELOG for 1.0.0-beta.2 release #886

Merged

iskiselev mentioned this pull request May 31, 2023

REQUEST: New membership for iskiselev open-telemetry/community#1517

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added CGroupv2 support into Docker Extensions #839

Added CGroupv2 support into Docker Extensions #839

anoopbabu29 commented Dec 16, 2022 •

edited

alexeypukhov Dec 19, 2022

anoopbabu29 Dec 19, 2022

utpilla commented Dec 19, 2022

iskiselev Dec 19, 2022

anoopbabu29 Dec 22, 2022

alexeypukhov Dec 23, 2022

alexeypukhov Dec 23, 2022

anoopbabu29 Dec 23, 2022

alexeypukhov Dec 23, 2022

alexeypukhov Dec 23, 2022 •

edited

anoopbabu29 Dec 23, 2022 •

edited

codecov bot commented Dec 23, 2022 •

edited

github-actions bot commented Dec 31, 2022

Kielek commented Jan 2, 2023

pellared commented Jan 2, 2023

iskiselev Jan 5, 2023

iskiselev Jan 5, 2023

anoopbabu29 Jan 5, 2023

iskiselev Jan 5, 2023

iskiselev Jan 5, 2023

anoopbabu29 Jan 5, 2023

iskiselev commented Jan 5, 2023

Kielek commented Jan 5, 2023

iskiselev Jan 6, 2023 •

edited

anoopbabu29 Jan 6, 2023

iskiselev left a comment

iskiselev commented Jan 9, 2023

Kielek commented Jan 9, 2023

Added CGroupv2 support into Docker Extensions #839

Added CGroupv2 support into Docker Extensions #839

Conversation

anoopbabu29 commented Dec 16, 2022 • edited

Which problem is this PR solving?

Short description of the changes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

utpilla commented Dec 19, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexeypukhov Dec 23, 2022 • edited

Choose a reason for hiding this comment

anoopbabu29 Dec 23, 2022 • edited

Choose a reason for hiding this comment

codecov bot commented Dec 23, 2022 • edited

Codecov Report

github-actions bot commented Dec 31, 2022

Kielek commented Jan 2, 2023

pellared commented Jan 2, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iskiselev commented Jan 5, 2023

Kielek commented Jan 5, 2023

iskiselev Jan 6, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iskiselev left a comment

Choose a reason for hiding this comment

iskiselev commented Jan 9, 2023

Kielek commented Jan 9, 2023

anoopbabu29 commented Dec 16, 2022 •

edited

alexeypukhov Dec 23, 2022 •

edited

anoopbabu29 Dec 23, 2022 •

edited

codecov bot commented Dec 23, 2022 •

edited

iskiselev Jan 6, 2023 •

edited