Cucumber-js parallel support

[xenjke]

Expected behaviour
Cucumber-js parallel test run report is uploaded and visible in Test Runs dashboard

Actual behaviour
Cucumber runtime gets terminated by an unhandled exception. Test report doesn't get uploaded when using cucumber-js --parallel argument:

TypeError: Cannot read properties of undefined (reading 'length')
    at <redacted>/node_modules/dd-trace/packages/datadog-instrumentations/src/cucumber.js:120:96
    at <anonymous>
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

Steps to reproduce

DD_API_KEY=redacted \
DD_CIVISIBILITY_AGENTLESS_ENABLED=true \
DD_ENV=redacted \
DD_LOGS_INJECTION=true \
DD_SERVICE=redacted \
DD_TAGS=redacted \
DD_TRACE_DEBUG=true \
LOG_LEVEL=debug \
NODE_OPTIONS="-r dd-trace/ci/init" \
npm run local:test:journey:cucumber -- --parallel 1 -d

Environment

• Operation system: N/A
• Node.js version: 16
• Tracer version: ?
• Agent version: ?
• Relevant library versions:
  • dd-trace: 3.15, 3.16, 3.18, 3.19, 4
  • cucumber-js: 9.1.2

If the Cucumber parallel capability isn't supported at the moment, it might be good to update the dd-trace compatibility page

[juan-fernandez]

hey @xenjke ! Thanks for the report. I'll check whether supporting parallel mode is a quick addition, otherwise I'll make sure the documentation is updated and that dd-trace does not make the test execution crash

[juan-fernandez]

#3156 should fix this issue. I'm disabling dd-trace when we detect we're in parallel mode because supporting it is not straightforward. I've added this to our backlog though, so we can eventually support it.

[xenjke]

@juan-fernandez Thank you for looking into this. Could you please provide an update on the estimated time of completion for this backlog item?

We are really interested in integrating this feature into our testing framework and are actively exploring opportunities to contribute towards its resolution. If you have any suggestions for external contributors, we would greatly appreciate any insights regarding the necessary requirements or support.

Thank you once again for your help.

[juan-fernandez]

hey @xenjke ! I can't provide an update right now, but we'll work on next quarter's planning soon, so I'll be able to tell you then :-).

As for contributions, they're more than welcome! The issue with parallel mode is usually that there needs to be interprocess communication between the tracer instances being initialised (an instance per process), because a suite can be "running" in a different process than a test, and the test needs info about the suite. This itself is not a huge deal, but since both parallel and serial mode need to work, the code becomes harder to navigate. This sadly makes this change not the friendliest first contribution to dd-trace-js :-/.

In any case, the fact that you're interested in this change is great feedback, so thank you!

[xenjke]

Happy new year team! As we kick off the new year and start planning, do you think we can expect this issue to be resolved in H1 2024?

[juan-fernandez]

Thanks! Happy new year to you too.

I can't provide an update right now, but we'll work on next quarter's planning soon, so I'll be able to tell you then :-)

I apologize, this slipped by 😅 .

do you think we can expect this issue to be resolved in H1 2024?

I'd like to tell you that we'll tackle it, but it's unlikely: cucumber usage is not very high and we don't have enough resources to give this any priority.

I'm more than happy to help you to contribute this though 😄