[cucumber/json-to-messages] Status from json file are transformed in the generated messages file

[cbohiptest]

Summary

Expected Behavior

The status of the steps in generated messages file should be the same as the status of the steps in the json file.

Current Behavior

When running the CLI described in https://www.npmjs.com/package/@cucumber/json-to-messages :
cat renewal.cucumber.json | ./node_modules/.bin/json-to-features > my_report.messages
the steps in the my_report.messages are incorrect one passed step become undefined, one passed step become skipped, one failed step become skipped.
[
files.zip
](url)

Version of @cucumber/json-to-messages is 6.0.0

[aslakhellesoy]

@cbohiptest can you please confirm the name and version of the tool that generated this JSON file?

[cbohiptest]

@cucumber/json-to-messages version 6.0.0

[aslakhellesoy]

Thanks. I meant the tool that generated the JSON file, not the messages file.

[cbohiptest]

I asked the user, waiting for his answer (different time zone).

[aslakhellesoy]

The json-to-messages tool expects every step in the JSON document to have a match property with a location property (unless the step was undefined). (This is in the various traverseStep functions in this library).

The step objects in the JSON file you attached are all passed, but they don't have a match property with a location. To illustrate, the first step should have looked like this:

{
  "arguments": [],
  "keyword": "Given ",
  "line": 14,
  "name": "Set up policy for renewal",
  "result": {
    "status": "passed",
    "duration": 1543000000
  },
  "match": {
    "location": "path/to/matching/step_definition.js:345"
  },
},

Since there is no match, the json-to-messages tool assumes that the step was undefined, even if the status field says it was passed. In other words, the JSON document has inconsistent information (it cannot be passed if it didn't match anything).

As far as I know, all Cucumber implementations correctly set the match object unless the step was undefined, which makes me wonder if this is a JSON file generated by a different tool than Cucumber? Didn't you mention Cypress @cbohiptest? Could it be that Cypress has its own (broken) Cucumber JSON generator?

@vincent-psarga - I wonder if we should change the logic so that we always derive the status from the status field.

[aslakhellesoy]

Looks like Cypress has its own Cucumber JSON generator that somehow uses Cucumber's JSON generator: https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/tree/master/lib/cukejson - I'd assume they just forgot to set the match field.

So two solutions:
a) Submit a bug report to Cypress (linking to this one) and wait for someone to fix it and make a new release
b) Improve this library to handle bad input

Any takers?

[vfuentes4713]

@aslakhellesoy Just to confirm yes we are using cypress v6.2.1

Presently we are defining our steps in a folder that matches the name of the feature file e.g

/renewal/happy-path.js
renewal.feature

but it sounds like we can also define them using steps_definition file i.e

"cypress-cucumber-preprocessor": {
    "step_definitions": "cypress/support/step_definitions/"
  }

I will give this a try but if our steps weren't being found then I assume we'd have a run time error. Will report back when I get some time to try this out.

[aslakhellesoy]

I don’t think the changes you are suggesting will make a difference for you. As I have tried to explain, the problem seems to be that Cypress is not generating the JSON file properly.

[aurelien-reeves]

Looks like Cypress has its own Cucumber JSON generator that somehow uses Cucumber's JSON generator: https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/tree/master/lib/cukejson - I'd assume they just forgot to set the match field.

So two solutions:
a) Submit a bug report to Cypress (linking to this one) and wait for someone to fix it and make a new release
b) Improve this library to handle bad input

Any takers?

I would suggest to do both.

If Cypress fix their resulting report, that would be great
But later we may encounter the same issue with another tool