IBM i CI failures #43509

richardlau · 2022-06-20T16:32:40Z

It looks like the IBM CI has been red for as long as we've got history: https://ci.nodejs.org/job/node-test-commit-ibmi/

From the latest CI run, https://ci.nodejs.org/job/node-test-commit-ibmi/798/#showFailuresLink, filtering out the tests already marked flaky we have these failures:

test.parallel/test-fs-readSync-position-validation

node:fs:747
  handleErrorFromBinding(ctx);
  ^

Error: EOVERFLOW: value too large for defined data type, read
    at Object.readSync (node:fs:747:3)
    at testValid (file:///home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-fs-readSync-position-validation.mjs:20:8)
    at file:///home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-fs-readSync-position-validation.mjs:60:3
    at ModuleJob.run (node:internal/modules/esm/module_job:197:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:549:24)
    at async loadESM (node:internal/process/esm_loader:83:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12) {
  errno: -127,
  syscall: 'read',
  code: 'EOVERFLOW'
}

Node.js v19.0.0-pre

test.parallel/test-icu-env

timeout

test.parallel/test-fs-read-position-validation

node:internal/process/esm_loader:89
    internalBinding('errors').triggerUncaughtException(
                              ^

[Error: EOVERFLOW: value too large for defined data type, read] {
  errno: -127,
  code: 'EOVERFLOW',
  syscall: 'read'
}

Node.js v19.0.0-pre

test.parallel/test-v8-serialize-leak

node:assert:399
    throw err;
    ^

AssertionError [ERR_ASSERTION]: before=0 after=0
    at Object.<anonymous> (/home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-v8-serialize-leak.js:21:3)
    at Module._compile (node:internal/modules/cjs/loader:1116:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1170:10)
    at Module.load (node:internal/modules/cjs/loader:994:32)
    at Module._load (node:internal/modules/cjs/loader:834:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:17:47 {
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: false,
  expected: true,
  operator: '=='
}

Node.js v19.0.0-pre

cc @nodejs/platform-ibmi

The text was updated successfully, but these errors were encountered:

richardlau · 2022-06-20T16:35:17Z

test.parallel/test-fs-readSync-position-validation

test.parallel/test-fs-read-position-validation

These two are variations on #36925 (comment). I'll open a PR to add EOVERFLOW to the allowed list of errors.

richardlau · 2022-06-20T16:40:59Z

test.parallel/test-v8-serialize-leak

I suspect this one is that IBM i doesn't (cannot?) report memory usage: #32043 (comment)

richardlau · 2022-06-20T18:57:42Z

@V-for-Vasili, since you volunteered to look, please take a look at test-icu-env on IBM i. It was added back in May by #42683 and appears to be timing out on IBM i, e.g. https://ci.nodejs.org/job/node-test-commit-ibmi/798/nodes=ibmi73-ppc64/testReport/junit/(root)/test/parallel_test_icu_env/

Some platforms may return `EOVERFLOW` errors instead of `EFBIG`. PR-URL: #43510 Refs: #42999 Refs: #43509 Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

V-for-Vasili · 2022-06-22T20:45:28Z

Looks like the whole test takes about 3m to complete on IBMi compared to about 11seconds on x64 linux.
All test assertions take comparable time to complete (there is not one that is significantly longer then the rest).
~~I think marking it SLOW could be an acceptable solution here.~~

Does not look like SLOW is an option for Release build, PR to skip: #43559

On IBM i rss memory information is not available and always returns `0`. PR-URL: #43511 Refs: #43509 Refs: libuv/libuv#2732 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

Some platforms may return `EOVERFLOW` errors instead of `EFBIG`. PR-URL: nodejs#43510 Refs: nodejs#42999 Refs: nodejs#43509 Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

On IBM i rss memory information is not available and always returns `0`. PR-URL: nodejs#43511 Refs: nodejs#43509 Refs: libuv/libuv#2732 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

richardlau · 2022-06-24T15:32:21Z

I edited the IBM i Jenkins job to increase the test timeout from the default 2 mins to 5 mins (via TEST_CI_ARGS="-t 300").

test-icu-env passed in https://ci.nodejs.org/job/node-test-commit-ibmi/803/nodes=ibmi73-ppc64/#showFailuresLink 🎉

It looks like we've gained two new failures in

V-for-Vasili · 2022-06-24T15:34:57Z

I assume this is not needed then #43559 if we intend to keep the timeout 5minutes on ibmi?

richardlau · 2022-06-24T15:37:12Z

I assume this is not needed then #43559 if we intend to keep the timeout 5minutes on ibmi?

Yeah, sorry about that. I'd be happy to keep the timeout on IBM i at 5 minutes (or even extend further if we find it necessary to do so).

richardlau · 2022-06-29T12:55:16Z

The two new failures are problems opening a test fixture:

09:24:45     NEW
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\n' +
09:24:45       'Error: Could not open history file.\n' +
09:24:45       'REPL session history will not be persisted.\n'
09:24:45     Failed test # 8
09:24:45     /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-repl-persistent-history.js:234
09:24:45               throw err;
09:24:45               ^
09:24:45     
09:24:45     AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
09:24:45     + actual - expected
09:24:45     
09:24:45     + '\n' +
09:24:45     +   'Error: Could not open history file.\n' +
09:24:45     +   'REPL session history will not be persisted.\n'
09:24:45     - undefined
09:24:45         at Writable.write [as _write] (/home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-repl-persistent-history.js:231:18)
09:24:45         at writeOrBuffer (node:internal/streams/writable:392:12)
09:24:45         at _write (node:internal/streams/writable:333:10)
09:24:45         at Writable.write (node:internal/streams/writable:337:10)
09:24:45         at [_writeToOutput] [as _writeToOutput] (node:internal/readline/interface:445:19)
09:24:45         at _writeToOutput (node:internal/repl/history:29:8)
09:24:45         at oninit (node:internal/repl/history:69:7)
09:24:45         at FSReqCallback.oncomplete (node:fs:201:23)
09:24:45         at FSReqCallback.callbackTrampoline (node:internal/async_hooks:130:17) {
09:24:45       generatedMessage: true,
09:24:45       code: 'ERR_ASSERTION',
09:24:45       actual: '\n' +
09:24:45         'Error: Could not open history file.\n' +
09:24:45         'REPL session history will not be persisted.\n',
09:24:45       expected: undefined,
09:24:45       operator: 'strictEqual'
09:24:45     }

09:24:45 not ok 2227 parallel/test-repl-persistent-history
09:24:45   ---
09:24:45   duration_ms: 2.1
09:24:45   severity: fail
09:24:45   exitcode: 1
09:24:45   stack: |-
09:24:45     NEW
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\n' +
09:24:45       'Persistent history support disabled. Set the NODE_REPL_HISTORY environment\n' +
09:24:45       'variable to a valid, user-writable path to enable.\n'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\r\n'
09:24:45     NEW
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\n' +
09:24:45       'Persistent history support disabled. Set the NODE_REPL_HISTORY environment\n' +
09:24:45       'variable to a valid, user-writable path to enable.\n'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\r\n'
09:24:45     NEW
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT "> 'you look fabulous today'"
09:24:45     INPUT '\x1B[28G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\r\n'
09:24:45     NEW
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '2'
09:24:45     INPUT '1'
09:24:45     INPUT '\r\n'
09:24:45     INPUT '21\n'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\r\n'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT "'"
09:24:45     INPUT '4'
09:24:45     INPUT '2'
09:24:45     INPUT "'"
09:24:45     INPUT '\r\n'
09:24:45     INPUT "'42'\n"
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\r\n'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\r\n'
09:24:45     NEW
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT "> '42'"
09:24:45     INPUT '\x1B[7G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> 21'
09:24:45     INPUT '\x1B[5G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\r\n'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT "> '42'"
09:24:45     INPUT '\x1B[7G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\r\n'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> 21'
09:24:45     INPUT '\x1B[5G'
09:24:45     INPUT '\r\n'
09:24:45     INPUT '21\n'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\r\n'
09:24:45     NEW
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT "> 'you look fabulous today'"
09:24:45     INPUT '\x1B[28G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT "> 'you look fabulous today'"
09:24:45     INPUT '\x1B[28G'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\r\n'
09:24:45     NEW
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\n' +
09:24:45       'Error: Could not open history file.\n' +
09:24:45       'REPL session history will not be persisted.\n'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\n' +
09:24:45       'Persistent history support disabled. Set the NODE_REPL_HISTORY environment\n' +
09:24:45       'variable to a valid, user-writable path to enable.\n'
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\r\n'
09:24:45     NEW
09:24:45     INPUT '\x1B[1G'
09:24:45     INPUT '\x1B[0J'
09:24:45     INPUT '> '
09:24:45     INPUT '\x1B[3G'
09:24:45     INPUT '\n' +
09:24:45       'Error: Could not open history file.\n' +
09:24:45       'REPL session history will not be persisted.\n'
09:24:45     Failed test # 8
09:24:45     /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-repl-persistent-history.js:234
09:24:45               throw err;
09:24:45               ^
09:24:45     
09:24:45     AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
09:24:45     + actual - expected
09:24:45     
09:24:45     + '\n' +
09:24:45     +   'Error: Could not open history file.\n' +
09:24:45     +   'REPL session history will not be persisted.\n'
09:24:45     - undefined
09:24:45         at Writable.write [as _write] (/home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-repl-persistent-history.js:231:18)
09:24:45         at writeOrBuffer (node:internal/streams/writable:392:12)
09:24:45         at _write (node:internal/streams/writable:333:10)
09:24:45         at Writable.write (node:internal/streams/writable:337:10)
09:24:45         at [_writeToOutput] [as _writeToOutput] (node:internal/readline/interface:445:19)
09:24:45         at _writeToOutput (node:internal/repl/history:29:8)
09:24:45         at oninit (node:internal/repl/history:69:7)
09:24:45         at FSReqCallback.oncomplete (node:fs:201:23)
09:24:45         at FSReqCallback.callbackTrampoline (node:internal/async_hooks:130:17) {
09:24:45       generatedMessage: true,
09:24:45       code: 'ERR_ASSERTION',
09:24:45       actual: '\n' +
09:24:45         'Error: Could not open history file.\n' +
09:24:45         'REPL session history will not be persisted.\n',
09:24:45       expected: undefined,
09:24:45       operator: 'strictEqual'
09:24:45     }
09:24:45     
09:24:45     Node.js v19.0.0-pre
09:24:45   ...
...
09:24:47 not ok 2235 parallel/test-repl-programmatic-history
09:24:48   ---
09:24:48   duration_ms: 2.119
09:24:48   severity: fail
09:24:48   exitcode: 1
09:24:48   stack: |-
09:24:48     NEW
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\n' +
09:24:48       'Persistent history support disabled. Set the NODE_REPL_HISTORY environment\n' +
09:24:48       'variable to a valid, user-writable path to enable.\n'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\r\n'
09:24:48     NEW
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\n' +
09:24:48       'Persistent history support disabled. Set the NODE_REPL_HISTORY environment\n' +
09:24:48       'variable to a valid, user-writable path to enable.\n'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\r\n'
09:24:48     NEW
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT "> 'you look fabulous today'"
09:24:48     INPUT '\x1B[28G'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\r\n'
09:24:48     NEW
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '2'
09:24:48     INPUT '1'
09:24:48     INPUT '\r\n'
09:24:48     INPUT '21\n'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\r\n'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT "'"
09:24:48     INPUT '4'
09:24:48     INPUT '2'
09:24:48     INPUT "'"
09:24:48     INPUT '\r\n'
09:24:48     INPUT "'42'\n"
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\r\n'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\r\n'
09:24:48     NEW
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT "> '42'"
09:24:48     INPUT '\x1B[7G'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> 21'
09:24:48     INPUT '\x1B[5G'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> 21'
09:24:48     INPUT '\x1B[5G'
09:24:48     INPUT '\r\n'
09:24:48     INPUT '21\n'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\r\n'
09:24:48     NEW
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT "> 'you look fabulous today'"
09:24:48     INPUT '\x1B[28G'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT "> 'you look fabulous today'"
09:24:48     INPUT '\x1B[28G'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\r\n'
09:24:48     NEW
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\n' +
09:24:48       'Error: Could not open history file.\n' +
09:24:48       'REPL session history will not be persisted.\n'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\n' +
09:24:48       'Persistent history support disabled. Set the NODE_REPL_HISTORY environment\n' +
09:24:48       'variable to a valid, user-writable path to enable.\n'
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\r\n'
09:24:48     NEW
09:24:48     INPUT '\x1B[1G'
09:24:48     INPUT '\x1B[0J'
09:24:48     INPUT '> '
09:24:48     INPUT '\x1B[3G'
09:24:48     INPUT '\n' +
09:24:48       'Error: Could not open history file.\n' +
09:24:48       'REPL session history will not be persisted.\n'
09:24:48     Failed test # 8
09:24:48     /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-repl-programmatic-history.js:231
09:24:48               throw err;
09:24:48               ^
09:24:48     
09:24:48     AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
09:24:48     + actual - expected
09:24:48     
09:24:48     + '\n' +
09:24:48     +   'Error: Could not open history file.\n' +
09:24:48     +   'REPL session history will not be persisted.\n'
09:24:48     - undefined
09:24:48         at Writable.write [as _write] (/home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-repl-programmatic-history.js:228:18)
09:24:48         at writeOrBuffer (node:internal/streams/writable:392:12)
09:24:48         at _write (node:internal/streams/writable:333:10)
09:24:48         at Writable.write (node:internal/streams/writable:337:10)
09:24:48         at [_writeToOutput] [as _writeToOutput] (node:internal/readline/interface:445:19)
09:24:48         at _writeToOutput (node:internal/repl/history:29:8)
09:24:48         at oninit (node:internal/repl/history:69:7)
09:24:48         at FSReqCallback.oncomplete (node:fs:201:23)
09:24:48         at FSReqCallback.callbackTrampoline (node:internal/async_hooks:130:17) {
09:24:48       generatedMessage: true,
09:24:48       code: 'ERR_ASSERTION',
09:24:48       actual: '\n' +
09:24:48         'Error: Could not open history file.\n' +
09:24:48         'REPL session history will not be persisted.\n',
09:24:48       expected: undefined,
09:24:48       operator: 'strictEqual'
09:24:48     }
09:24:48     
09:24:48     Node.js v19.0.0-pre
09:24:48   ...

richardlau · 2022-06-29T12:57:28Z

I logged into the machine and was able to see that, for some reason, one of the test fixtures was not owned by the iojs user running the tests:

bash-5.1$ ls -al /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/fixtures/
total 2608
drwxrwsrwx 51 iojs   0 126976 Jun 20 12:05 .
drwxrwsrwx 30 iojs   0  24576 Jun 29 08:32 ..
-rw-r--r--  1 nodejs 0      0 Jun 29 08:21 .empty-hidden-repl-history-file
-rw-rw-rw-  1 iojs   0      0 Jun  2 03:04 .empty-repl-history-file
-rw-rw-rw-  1 iojs   0     40 Jun  2 03:04 .node_repl_history
drwxrwsrwx  2 iojs   0   8192 Jun  2 03:04 0-dns

I'm not entirely sure how that has happened -- I've changed the owner of that file back to iojs and re-running: https://ci.nodejs.org/job/node-test-commit-ibmi/809/nodes=ibmi73-ppc64/

richardlau · 2022-06-29T17:11:02Z

re-running: https://ci.nodejs.org/job/node-test-commit-ibmi/809/nodes=ibmi73-ppc64/

Good news is the repl test failures have gone away. Bad news is that this run has yet another failure 😢.

https://ci.nodejs.org/job/node-test-commit-ibmi/809/nodes=ibmi73-ppc64/testReport/junit/(root)/test/parallel_test_http_pipeline_requests_connection_leak/

node:events:515
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:217:20)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -73,
  code: 'ECONNRESET',
  syscall: 'read'
}

Node.js v19.0.0-pre

Trying again: https://ci.nodejs.org/job/node-test-commit-ibmi/810/nodes=ibmi73-ppc64/

V-for-Vasili · 2022-06-29T17:25:04Z

This error message looks like the one from ECONNRESET test failures a while back. Were PTFs MF69727 and MF69652
applied to this specific node (test-iinthecloud-ibmi73-ppc64_be-1) by Jesse?

richardlau · 2022-06-29T17:51:28Z

This error message looks like the one from ECONNRESET test failures a while back. Were PTFs MF69727 and MF69652 applied to this specific node (test-iinthecloud-ibmi73-ppc64_be-1) by Jesse?

I have no idea. @ThePrez were the PTF's applied to the iinthecloud machines as well as the rzkh one?

ThePrez · 2022-06-29T19:41:20Z

Sorry, @richardlau et al.
Looks like the iinthecloud patch is still pending. The fix is only on the rzkh one. I contacted the system team and will update here when the fix is applied

Some platforms may return `EOVERFLOW` errors instead of `EFBIG`. PR-URL: #43510 Refs: #42999 Refs: #43509 Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

On IBM i rss memory information is not available and always returns `0`. PR-URL: #43511 Refs: #43509 Refs: libuv/libuv#2732 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

Some platforms may return `EOVERFLOW` errors instead of `EFBIG`. PR-URL: #43510 Refs: #42999 Refs: #43509 Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

On IBM i rss memory information is not available and always returns `0`. PR-URL: #43511 Refs: #43509 Refs: libuv/libuv#2732 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

Some platforms may return `EOVERFLOW` errors instead of `EFBIG`. PR-URL: #43510 Refs: #42999 Refs: #43509 Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

On IBM i rss memory information is not available and always returns `0`. PR-URL: #43511 Refs: #43509 Refs: libuv/libuv#2732 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

Trott · 2022-08-07T16:25:45Z

Still lots of IBM i CI failures. Here's today's daily run: https://ci.nodejs.org/job/node-test-commit-ibmi/849/nodes=ibmi73-ppc64/#showFailuresLink

It seems to be the difference between a red daily run and a yellow daily run. Should we consider removing IBM i from the daily run until things are in a better state? Or maybe moving it to its own daily run so it doesn't affect the daily results for overall?

richardlau · 2022-08-08T17:02:25Z

@ThePrez Any news on APAR for the iinthecloud machines?

ThePrez · 2022-08-09T14:56:32Z

@richardlau unfortunately not. They have been unresponsive.
They did just get acquired (or maybe merged with) another company, so I am not sure if the email is broken, they are underwater with the merger, no longer have interest, or all of the above. Can we exclude them from the builds for now? The rzkh system should have the fixes

richardlau · 2022-08-11T16:42:02Z

Still lots of IBM i CI failures. Here's today's daily run: https://ci.nodejs.org/job/node-test-commit-ibmi/849/nodes=ibmi73-ppc64/#showFailuresLink

There's only one failure there that isn't already marked flaky -- test-http-pipeline-requests-connection-leak. This was evaluated earlier in this issue (#43509 (comment)) as being related to a set of already marked flaky tests (#39683). There are OS PTFs (i.e. patches) available to fix the underlying issue but as noted we only have them on the rzkh IBM i CI machine and haven't yet been able to apply them to the iinthecloud machines (the Build WG are unable to do this).

I've opened a PR to mark test-http-pipeline-requests-connection-leak as flaky on IBM i: #44215
FTR I forced a CI run onto the rzkh machine which has thrown up a different, much smaller, set of failures: https://ci.nodejs.org/job/node-test-commit-ibmi/855/nodes=ibmi73-ppc64/
(the rzkh machine appears unreachable today so I haven't been able to repeat).

Mark `test-http-pipeline-requests-connection-leak` flaky on IBM i. PR-URL: #44215 Refs: #43509 Refs: #39683 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Darshan Sen <raisinten@gmail.com>

richardlau · 2022-08-15T14:09:17Z

With #44215 we're down to one test that is failing the IBM i CI build:

https://ci.nodejs.org/job/node-test-commit-ibmi/862/nodes=ibmi73-ppc64/console

10:21:56 not ok 3714 sequential/test-tls-psk-client # TODO : Fix flaky test
10:26:56   ---
10:26:56   duration_ms: 300.226
10:26:56   severity: fail
10:26:56   exitcode: -15
10:26:56   stack: |-
10:26:56     timeout
10:26:56     Failed: Timed out
10:26:56   ...

This test is already marked flaky on IBM i

node/test/sequential/sequential.status

Lines 30 to 31 in d7f1934

    
           # https://github.com/nodejs/node/issues/39683 
        
           test-tls-psk-client: PASS, FLAKY

but it looks like the test runner treats timed out tests as failures (severity: fail) instead of flakes (severity: flaky).

Mark `test-http-pipeline-requests-connection-leak` flaky on IBM i. PR-URL: #44215 Refs: #43509 Refs: #39683 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Darshan Sen <raisinten@gmail.com>

Mark `test-http-pipeline-requests-connection-leak` flaky on IBM i. PR-URL: nodejs#44215 Refs: nodejs#43509 Refs: nodejs#39683 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Darshan Sen <raisinten@gmail.com>

richardlau · 2022-09-29T14:35:36Z

I inadvertently introduced new failures with #44148 that are being addressed by #44810.

I've opened #44821 for the remaining issue.

Some platforms may return `EOVERFLOW` errors instead of `EFBIG`. PR-URL: nodejs/node#43510 Refs: nodejs/node#42999 Refs: nodejs/node#43509 Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

On IBM i rss memory information is not available and always returns `0`. PR-URL: nodejs/node#43511 Refs: nodejs/node#43509 Refs: libuv/libuv#2732 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

Mark `test-http-pipeline-requests-connection-leak` flaky on IBM i. PR-URL: #44215 Refs: #43509 Refs: #39683 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Darshan Sen <raisinten@gmail.com>

Mark `test-http-pipeline-requests-connection-leak` flaky on IBM i. PR-URL: nodejs/node#44215 Refs: nodejs/node#43509 Refs: nodejs/node#39683 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Darshan Sen <raisinten@gmail.com>

richardlau added the ibm i Issues and PRs related to the IBM i platform. label Jun 20, 2022

This was referenced Jun 20, 2022

test: allow EOVERFLOW errors in fs position tests #43510

Merged

test: skip test-v8-serialize-leak on IBM i #43511

Merged

V-for-Vasili mentioned this issue Jun 24, 2022

test: skip test-icu-env on IBM i #43559

Closed

richardlau mentioned this issue Aug 11, 2022

test: mark connection leak test flaky on IBM i #44215

Merged

richardlau mentioned this issue Sep 29, 2022

sequential/test-tls-psk-client fails on IBM i #44821

Open

richardlau closed this as completed Sep 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IBM i CI failures #43509

IBM i CI failures #43509

richardlau commented Jun 20, 2022 •

edited

richardlau commented Jun 20, 2022

richardlau commented Jun 20, 2022

richardlau commented Jun 20, 2022

V-for-Vasili commented Jun 22, 2022 •

edited

richardlau commented Jun 24, 2022

V-for-Vasili commented Jun 24, 2022

richardlau commented Jun 24, 2022

richardlau commented Jun 29, 2022

richardlau commented Jun 29, 2022

richardlau commented Jun 29, 2022

V-for-Vasili commented Jun 29, 2022

richardlau commented Jun 29, 2022

ThePrez commented Jun 29, 2022

Trott commented Aug 7, 2022

richardlau commented Aug 8, 2022

ThePrez commented Aug 9, 2022

richardlau commented Aug 11, 2022

richardlau commented Aug 15, 2022

richardlau commented Sep 29, 2022

IBM i CI failures #43509

IBM i CI failures #43509

Comments

richardlau commented Jun 20, 2022 • edited

test.parallel/test-fs-readSync-position-validation

test.parallel/test-icu-env

test.parallel/test-fs-read-position-validation

test.parallel/test-v8-serialize-leak

richardlau commented Jun 20, 2022

richardlau commented Jun 20, 2022

richardlau commented Jun 20, 2022

V-for-Vasili commented Jun 22, 2022 • edited

richardlau commented Jun 24, 2022

V-for-Vasili commented Jun 24, 2022

richardlau commented Jun 24, 2022

richardlau commented Jun 29, 2022

richardlau commented Jun 29, 2022

richardlau commented Jun 29, 2022

V-for-Vasili commented Jun 29, 2022

richardlau commented Jun 29, 2022

ThePrez commented Jun 29, 2022

Trott commented Aug 7, 2022

richardlau commented Aug 8, 2022

ThePrez commented Aug 9, 2022

richardlau commented Aug 11, 2022

richardlau commented Aug 15, 2022

richardlau commented Sep 29, 2022

richardlau commented Jun 20, 2022 •

edited

V-for-Vasili commented Jun 22, 2022 •

edited