ncu-ci commit TypeError: Cannot read properties of null (reading '1') #639

F3n67u · 2022-07-04T15:38:07Z

Reproduce

$ ncu-ci commit 54550                      
--------------------------------------------------------------------------------
[1/1] Running COMMIT: 54550
--------------------------------------------------------------------------------
✔  Build data downloaded
⠏ Querying API for job/node-test-binary-windows-js-suites/RUN_SUBSET=2,nodes=win10-arm64-COMPILED_BY-vs2019-arm64/13778/TypeError: Cannot read properties of null (reading '1')
    at new JSTestFailure (file:///usr/local/lib/node_modules/node-core-utils/lib/ci/ci_failure_parser.js:78:62)
    at file:///usr/local/lib/node_modules/node-core-utils/lib/ci/ci_failure_parser.js:168:16
    at Array.map (<anonymous>)
    at filter (file:///usr/local/lib/node_modules/node-core-utils/lib/ci/ci_failure_parser.js:167:29)
    at CIFailureParser.parse (file:///usr/local/lib/node_modules/node-core-utils/lib/ci/ci_failure_parser.js:296:22)
    at TestRun.parseConsoleText (file:///usr/local/lib/node_modules/node-core-utils/lib/ci/build-types/job.js:93:26)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 13)
    at async CommitBuild.getResults (file:///usr/local/lib/node_modules/node-core-utils/lib/ci/build-types/commit_build.js:106:25)
    at async JobCommand.drain (file:///usr/local/lib/node_modules/node-core-utils/bin/ncu-ci.js:352:7)

Expected behavior

No Error happened, could process job/node-test-commit/54550/ normally

Actual behavior

TypeError thrown

Additional Information

Refs: nodejs/reliability#316

Jenkins job URL:
https://ci.nodejs.org/job/node-test-commit-osx-arm/6298/
https://ci.nodejs.org/job/node-test-commit/54550/
https://ci.nodejs.org/job/node-test-pull-request/44957/

The text was updated successfully, but these errors were encountered:

aduh95 · 2022-07-04T22:37:28Z

The CI has been locked in preparation for the security release, I think the error is the expected behavior.

richardlau · 2022-07-04T22:41:56Z

This was occuring at the end of last week before the CI was locked down, see nodejs/reliability#316.

F3n67u · 2022-07-11T09:59:04Z

After some debugging, I found the following additional info, but still don't figure out why the test console text is not behaving as expected.

Problem Jenkins job URL: https://ci.nodejs.org/job/node-test-commit-osx-arm/6298/
from https://ci.nodejs.org/job/node-test-commit/54550/
from https://ci.nodejs.org/job/node-test-pull-request/44957/

Problem Console Text:
https://ci.nodejs.org/job/node-test-commit-osx-arm/nodes=osx11/6298/consoleText
https://ci.nodejs.org/job/node-test-commit-osx-arm/6298/nodes=osx11/consoleFull

Pr where the job trigger:
nodejs/node#43549

The console text which cause parse error:

outline = not ok 3 - sync fail todo # TODO
outline =   ---
outline =   duration_ms: 0.000040417
outline =   failureType: 'testCodeFailure'
outline =   error: 'thrown from sync fail todo'
outline =   code: 'ERR_TEST_FAILURE'
outline =   stack: |-
outline =     Object.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/message/test_runner_desctibe_it.js:17:9)
outline =     ItTest.runInAsyncScope (node:async_hooks:203:9)
outline =     ItTest.run (node:internal/test_runner/test:338:15)
outline =     Test.processPendingSubtests (node:internal/test_runner/test:158:27)
outline =     ItTest.postRun (node:internal/test_runner/test:390:19)
outline =     ItTest.run (node:internal/test_runner/test:352:10)
outline =     async Test.processPendingSubtests (node:internal/test_runner/test:158:7)
outline =   ...

More context:

length differs.
expect=512
actual=513
patterns:
pattern = ^TAP\ version\ 13$
pattern = ^\#\ Subtest:\ sync\ pass\ todo$
pattern = ^ok\ 1\ \-\ sync\ pass\ todo\ \#\ TODO$

<...omit rest pattern text>

outlines:
outline = TAP version 13
outline = # Subtest: sync pass todo
outline = ok 1 - sync pass todo # TODO
outline =   ---
outline =   duration_ms: 0.001743083
outline =   ...
outline = # Subtest: sync pass todo with message
outline = ok 2 - sync pass todo with message # TODO this is a passing todo
outline =   ---
outline =   duration_ms: 0.000014917
outline =   ...
outline = # Subtest: sync fail todo
outline = not ok 3 - sync fail todo # TODO
outline =   ---
outline =   duration_ms: 0.000040417
outline =   failureType: 'testCodeFailure'
outline =   error: 'thrown from sync fail todo'
outline =   code: 'ERR_TEST_FAILURE'
outline =   stack: |-
outline =     Object.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/message/test_runner_desctibe_it.js:17:9)
outline =     ItTest.runInAsyncScope (node:async_hooks:203:9)
outline =     ItTest.run (node:internal/test_runner/test:338:15)
outline =     Test.processPendingSubtests (node:internal/test_runner/test:158:27)
outline =     ItTest.postRun (node:internal/test_runner/test:390:19)
outline =     ItTest.run (node:internal/test_runner/test:352:10)
outline =     async Test.processPendingSubtests (node:internal/test_runner/test:158:7)
outline =   ...

<...omit rest outline text>

F3n67u · 2022-07-11T10:01:55Z

Problem points:

There is no severity attribute.

The weird outline = prefix.

@MoLow do you have any idea why?

F3n67u · 2022-07-11T10:27:00Z

pattern length and outline length differs.
pattern length is 512
outline length is 513

the diff is from last part.

pattern is:

pattern = ^\#\ tests\ 54$
pattern = ^\#\ pass\ 23$
pattern = ^\#\ fail\ 17$
pattern = ^\#\ skipped\ 9$
pattern = ^\#\ todo\ 5$
pattern = ^\#\ duration_ms\ .*$

outline is:

outline = # tests 54
outline = # pass 23
outline = # fail 17
outline = # cancelled 0
outline = # skipped 9
outline = # todo 5
outline = # duration_ms 4.798118667

the pattern has not cancelled but the outline has cancelled. this is why length differs print and cause the parse error.

F3n67u · 2022-07-11T11:03:39Z

I know what is the problem now:

test_runner_desctibe_it has a snapshot: test_runner_desctibe_it.out, but the actual output is not match the snapshot
the test_runner_desctibe_it fail and print a log that says the pattern and outline is not match
the outline is a test output, ncu-ci think it is a real test output and tries to parse it and resulting in a parse failure

MoLow · 2022-07-11T11:13:03Z

@F3n67u anything I can do to help?
I guess we can either change the output of tools/test.py in case of a output that contains tap
or we can change the code in this repo to accept the malformed output.
Il try playing around with both options

MoLow · 2022-07-11T11:13:38Z

Oh thanks for fixing it :)

F3n67u · 2022-07-11T11:15:09Z

@F3n67u anything I can do to help? I guess we can either change the output of tools/test.py in case of a output that contains tap or we can change the code in this repo to accept the malformed output. Il try playing around with both options

I figure out how to fix this problem. I made a pr to fix this.

richardlau · 2022-07-11T12:02:42Z

FWIW we may have a similar issue in https://github.com/nodejs/tap2junit when parsing failures from test_runner_desctibe_it, nodejs/node#43773

MoLow · 2022-07-13T04:29:37Z

Seems like the issue can be closed: nodejs/reliability#321

F3n67u · 2022-07-13T05:38:09Z

fix: parse ci failure error #640

no worry. this issue will closed by #640

F3n67u · 2022-07-30T03:09:38Z

Update CI reliability job stop working again, review wanted for #640.
https://github.com/nodejs/reliability/runs/7587812485?check_suite_focus=true

F3n67u mentioned this issue Jul 4, 2022

No more CI Reliability report since 4 days ago nodejs/reliability#316

Closed

F3n67u mentioned this issue Jul 11, 2022

fix: parse ci failure error #640

Merged

richardlau mentioned this issue Jul 11, 2022

test_runner_describe_it failures break TAP output nodejs/node#43773

Closed

MoLow mentioned this issue Jul 12, 2022

Fix parsing yaml blocks nodejs/tap2junit#31

Merged

aduh95 closed this as completed in #640 Jul 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ncu-ci commit TypeError: Cannot read properties of null (reading '1') #639

ncu-ci commit TypeError: Cannot read properties of null (reading '1') #639

F3n67u commented Jul 4, 2022 •

edited

aduh95 commented Jul 4, 2022

richardlau commented Jul 4, 2022

F3n67u commented Jul 11, 2022 •

edited

F3n67u commented Jul 11, 2022

F3n67u commented Jul 11, 2022

F3n67u commented Jul 11, 2022 •

edited

MoLow commented Jul 11, 2022

MoLow commented Jul 11, 2022

F3n67u commented Jul 11, 2022

richardlau commented Jul 11, 2022

MoLow commented Jul 13, 2022

F3n67u commented Jul 13, 2022

F3n67u commented Jul 30, 2022

ncu-ci commit TypeError: Cannot read properties of null (reading '1') #639

ncu-ci commit TypeError: Cannot read properties of null (reading '1') #639

Comments

F3n67u commented Jul 4, 2022 • edited

Reproduce

Expected behavior

Actual behavior

Additional Information

aduh95 commented Jul 4, 2022

richardlau commented Jul 4, 2022

F3n67u commented Jul 11, 2022 • edited

F3n67u commented Jul 11, 2022

F3n67u commented Jul 11, 2022

F3n67u commented Jul 11, 2022 • edited

MoLow commented Jul 11, 2022

MoLow commented Jul 11, 2022

F3n67u commented Jul 11, 2022

richardlau commented Jul 11, 2022

MoLow commented Jul 13, 2022

F3n67u commented Jul 13, 2022

F3n67u commented Jul 30, 2022

F3n67u commented Jul 4, 2022 •

edited

F3n67u commented Jul 11, 2022 •

edited

F3n67u commented Jul 11, 2022 •

edited