Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default image for autoinstrumentation dotnet is not setting the right path for Startup Hook #2728

Open
juanchovelezpro opened this issue May 3, 2024 · 0 comments

Comments

@juanchovelezpro
Copy link

Describe the bug
I was trying to use dotnet instrumentation, I created this Instrumentation object

apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: dotnet-instrumentation
spec:
  exporter:
    endpoint: http://adot-col-otlp-ingest-collector.opentelemetry-operator-system.svc.cluster.local:4317
  sampler:
    type: parentbased_traceidratio
    argument: "0.25"
  dotnet:
    env:
      - name: OTEL_EXPORTER_OTLP_ENDPOINT
        value: http://adot-col-otlp-ingest-collector.opentelemetry-operator-system.svc.cluster.local:4318

Then when I annotate a dotnet workload with instrumentation.opentelemetry.io/inject-dotnet: true I get the following error:

Unhandled exception. System.ArgumentException: Startup hook assembly '/otel-auto-instrumentation-dotnet/net/OpenTelemetry.AutoInstrumentation.StartupHook.dll' failed to load. See inner exception for details.
 ---> System.IO.FileNotFoundException: Could not load file or assembly '/otel-auto-instrumentation-dotnet/net/OpenTelemetry.AutoInstrumentation.StartupHook.dll'. The system cannot find the file specified.

File name: '/otel-auto-instrumentation-dotnet/net/OpenTelemetry.AutoInstrumentation.StartupHook.dll'
   at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
   at System.StartupHookProvider.CallStartupHook(StartupHookNameOrPath startupHook)
   --- End of inner exception stack trace ---
   at System.StartupHookProvider.CallStartupHook(StartupHookNameOrPath startupHook)
   at System.StartupHookProvider.ProcessStartupHooks(String diagnosticStartupHooks)

Debugging in a different pod I found that the initContainer using the image
public.ecr.aws/aws-observability/adot-operator-autoinstrumentation-dotnet:1.2.0 is not copying the autoinstrumentation in the right path. This image is copying in /otel-auto-instrumentation-dotnet/autoinstrumentation/* but the environment variables that require this path are defined with:
image
As you can see the autoinstrumentation folder is not defined in these environment variables.
But when I use the image ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-dotnet:1.6.0 is copying all in /otel-auto-instrumentation-dotnet/* and this is the right path that is in the environment variables.

Not sure why the adot image for dotnet autoinstrumentation is also copying the folder autoinstrumentation under the folder otel-auto-instrumentation-dotnet causing the error to find the assemblies. But using the other image is copying all the files in the path that is specified in the environment variables. I saw that both images execute the same command:
cp -a /autoinstrumentation/. /otel-auto-instrumentation-dotnet

But the adot image is adding the folder autoinstrumentation under otel-auto-instrumentation-dotnet folder.

Steps to reproduce
I installed the ADOT using eksctl adding this

- name: adot
  version: v0.94.1-eksbuild.1
  resolveConflicts: overwrite
  configurationValues: |-
    tolerations:
    - operator: Exists
    collector:
      otlpIngest:
        serviceAccount:
          annotations:
            eks.amazonaws.com/role-arn: arn:aws:iam::<MY_ACCOUNT_ID>:role/adot-col-otlp-ingest-dev1
        pipelines:
          traces:
            xray:
              enabled: true

Then I created the Instrumentation object:

apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: dotnet-instrumentation
spec:
  exporter:
    endpoint: http://adot-col-otlp-ingest-collector.opentelemetry-operator-system.svc.cluster.local:4317
  sampler:
    type: parentbased_traceidratio
    argument: "0.25"
  dotnet:
    env:
      - name: OTEL_EXPORTER_OTLP_ENDPOINT
        value: http://adot-col-otlp-ingest-collector.opentelemetry-operator-system.svc.cluster.local:4318`

Finally I annotate the workload adding (In the PodSpec template of the Deployment):
instrumentation.opentelemetry.io/inject-dotnet: true

What did you expect to see?
The official ADOT image for dotnet autoinstrumentation should copy the autoinstrumentation assemblies in /otel-auto-instrumentation-dotnet

What did you see instead?
All the autoinstrumentation files were copied in /otel-auto-instrumentation-dotnet/autoinstrumentation, causing the error because the environment variables added by the operator don't match with this path.

Environment
EKS - Kubernetes version 1.28
EKS Addon ADOT Version: v0.94.1-eksbuild.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant