Exporting traces to a collector not working

[ghirsch1]

I'm trying to set up tracing on a node service for the first time. I'm trying something basic like the code below (based on one of the examples), and the auto-instrumentation seems to be working based on the fact that I'm seeing reasonable output in the console. However, the exporter to the collector seems broken, and I'm seeing the following error. Could anyone help me figure out what I'm doing wrong?

I am also confident that the collector itself does work, since I am able to send traces to it from a different service written in Go.

Thanks in advance for any help!

(node:72342) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'replace' of undefined
    at ServiceClient.clientMethodTrace [as export] (/Users/g/server/node_modules/@opentelemetry/plugin-grpc/src/grpc.ts:389:44)
    at CollectorTraceExporter.send [as _send] (/Users/g/server/node_modules/@opentelemetry/exporter-collector-grpc/src/util.ts:82:35)
    at /Users/g/server/node_modules/@opentelemetry/exporter-collector-grpc/src/CollectorExporterNodeBase.ts:72:12
    at new Promise (<anonymous>)
    at CollectorTraceExporter._sendPromise (/Users/g/server/node_modules/@opentelemetry/exporter-collector-grpc/src/CollectorExporterNodeBase.ts:57:21)
    at Immediate.<anonymous> (/Users/g/server/node_modules/@opentelemetry/exporter-collector-grpc/src/CollectorExporterNodeBase.ts:106:14)
    at processImmediate (internal/timers.js:461:21)
    at process.topLevelDomainCallback (domain.js:138:15)
    at process.callbackTrampoline (internal/async_hooks.js:124:14)

tracing.ts (I've also tried some small changes like specifying the url and service name for the exporter, or using HTTP instead of gRPC)

import { SimpleSpanProcessor, ConsoleSpanExporter } from '@opentelemetry/tracing';
import { NodeTracerProvider } from '@opentelemetry/node';
import { CollectorTraceExporter } from '@opentelemetry/exporter-collector-grpc';


const provider = new NodeTracerProvider();
provider.addSpanProcessor(new SimpleSpanProcessor(new CollectorTraceExporter()));
provider.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()));
provider.register();

index.ts

import * as express from "express";

const app = express();
app.get("/_healthcheck", (req, res, next) => {
  res.sendStatus(200);
});

// Start the server
app.listen(PORT, () => {
  console.log(`Listening on port ${PORT}...`);
});

example traces showing up in console

{
  traceId: '458ab26b6429b6259c52bf7b76096f1c',
  parentId: '4329c9f0332c0852',
  name: 'middleware - query',
  id: '88262cc4103251dc',
  kind: 0,
  timestamp: 1610475216345450,
  duration: 367,
  attributes: {
    component: 'express',
    'express.name': 'query',
    'express.type': 'middleware'
  },
  status: { code: 1 },
  events: []
}
{
  traceId: '458ab26b6429b6259c52bf7b76096f1c',
  parentId: '4329c9f0332c0852',
  name: 'middleware - expressInit',
  id: '8508da3ad1144a88',
  kind: 0,
  timestamp: 1610475216348011,
  duration: 371,
  attributes: {
    component: 'express',
    'express.name': 'expressInit',
    'express.type': 'middleware'
  },
  status: { code: 1 },
  events: []
}
{
  traceId: '458ab26b6429b6259c52bf7b76096f1c',
  parentId: '4329c9f0332c0852',
  name: 'middleware - <anonymous>',
  id: 'c87db1386fc7cced',
  kind: 0,
  timestamp: 1610475216348764,
  duration: 876,
  attributes: {
    component: 'express',
    'express.name': '<anonymous>',
    'express.type': 'middleware'
  },
  status: { code: 1 },
  events: []
}
{
  traceId: '458ab26b6429b6259c52bf7b76096f1c',
  parentId: '4329c9f0332c0852',
  name: 'request handler - /_healthcheck',
  id: '3a0c633b2f7170f5',
  kind: 0,
  timestamp: 1610475216350102,
  duration: 36,
  attributes: {
    component: 'express',
    'http.route': '/_healthcheck',
    'express.name': '/_healthcheck',
    'express.type': 'request_handler'
  },
  status: { code: 1 },
  events: []
}
{
  traceId: '458ab26b6429b6259c52bf7b76096f1c',
  parentId: undefined,
  name: 'GET /_healthcheck',
  id: '4329c9f0332c0852',
  kind: 1,
  timestamp: 1610475216343631,
  duration: 10480,
  attributes: {
    'http.url': 'http://localhost:6856/_healthcheck',
    'http.host': 'localhost:6856',
    'net.host.name': 'localhost',
    'http.method': 'GET',
    'http.route': '/_healthcheck',
    'http.target': '/_healthcheck',
    'http.user_agent': 'curl/7.64.1',
    'http.flavor': '1.1',
    'net.transport': 'IP.TCP',
    'net.host.ip': '::1',
    'net.host.port': 6856,
    'net.peer.ip': '::1',
    'net.peer.port': 54462,
    'http.status_code': 200,
    'http.status_text': 'OK'
  },
  status: { code: 0 },
  events: []
}

[dyladan]

Can you please provide all installed versions of OpenTelemetry modules and the collector? We have seen errors like this in the past when someone is using a different version of the exporter than the tracing module for instance.

[dyladan]

I would prefer to consolidate here

[ghirsch1]

When I tried updating the various otel-js versions back to 0.14.0, I no longer get that type error. So perhaps it was an issue with a different dependency?

Apologies for the confusion!

[dyladan]

hmm. when in doubt it can't hurt to blow away your node_modules and reinstall. in any case, i'm glad the issue is solved

[obecny]

it should work with latest 0.14 for core repo and 0.12 for contrib (talking about js). But it is a collector version that I'm worry about - otel/opentelemetry-collector-contrib:0.17.0. The latest that we tested was otel/opentelemetry-collector:0.16.0

[haseeb-aziz]

hello I'm getting this error.
rpc error: code = Unavailable desc = connection error: desc = "error reading server preface: http2: frame too large\

my scenario is, I want to send prometheus metrics to uptrace using opentelemetery collector. All these services I deployed on eks cluster using helm chart.
please guide where I've done mistake.
Looking forward to your response.