Parallel e2e tests: Only start one instance of Parity, Bitcoind and one lnd per actor

[thomaseizinger]

By utilizing file-locks on directories, we can ensure that any number
of parallel test suites only ever get to run one instance of
the ledger. The remaining test suites will take the config and
pass that to the tests.

We store the PID of each ledger in a file inside the lock directory,
this also includes a sub-process that we spawn for the Bitcoin miner.
This is important, because same as with the other nodes, this process
must keep going until the last test executed.

To cleanup our environment, we kill all processes with PIDs that we
find in those PID files before and after we execute the tests, just
in case if we for some reason did not exit cleanly and failed to
cleanup after ourselves.

What works:

• Interrupting the test suite with CTRL+C cleans the enviornment
• Running tests in parallel
• Running an individual test
• Nodes shut down after a successful run

What doesn't work

• A failing test will not clean up the environment

[mergify]

Are you sure the changelog does not need updating?

[bonomat]

Status from my machine:
make e2e failed:

Details (click here)

(cd ./api_tests; yarn install; yarn test)
yarn install v1.22.0
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
[3/5] 🚚  Fetching packages...
[4/5] 🔗  Linking dependencies...
[5/5] 🔨  Building fresh packages...
✨  Done in 12.41s.
yarn run v1.22.0
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 2.17s
$ jest
 PASS  tests/lightning_routes.ts (30.587s)
(node:71803) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:71804) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/rfc003_schema.ts (38.758s)
 PASS  tests/peers_using_ip.ts (9.732s)
(node:71802) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/multiple_peers.ts
 PASS  tests/sanity.ts (43.259s)
 FAIL  tests/bitcoin_ethereum.ts
  ● Test suite failed to run

    Lock file is already being held

      at node_modules/proper-lockfile/lib/lockfile.js:68:47
      at callback (node_modules/graceful-fs/polyfills.js:295:20)

 FAIL  tests/ethereum_bitcoin.ts
  ● Test suite failed to run

    Lock file is already being held

      at node_modules/proper-lockfile/lib/lockfile.js:68:47
      at callback (node_modules/graceful-fs/polyfills.js:295:20)

 FAIL  tests/lnd_sanity.ts
  ● Test suite failed to run

    timed out after 60000ms

      51 |         const id = setTimeout(() => {
      52 |             clearTimeout(id);
    > 53 |             reject(new Error(`timed out after ${ms}ms`));
         |                    ^
      54 |         }, ms);
      55 |     });
      56 | 

      at Timeout.<anonymous> (src/utils.ts:53:20)

Test Suites: 3 failed, 5 passed, 8 total
Tests:       22 passed, 22 total
Snapshots:   0 total
Time:        106.075s
Ran all test suites.
A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
make: *** [e2e] Error 1

After killing parity and bitcoind the first run yarn test passed

Details (click here)

yarn test
yarn run v1.22.0
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 0.83s
$ jest
(node:72096) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/rfc003_schema.ts (19.83s)
 PASS  tests/lightning_routes.ts (26.319s)
 PASS  tests/peers_using_ip.ts (9.061s)
(node:72095) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/sanity.ts (33.737s)
 PASS  tests/multiple_peers.ts (9.42s)
(node:72097) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/lnd_sanity.ts (82.099s)
 PASS  tests/ethereum_bitcoin.ts (211.457s)
 PASS  tests/bitcoin_ethereum.ts (469.823s)

Test Suites: 8 passed, 8 total
Tests:       44 passed, 44 total
Snapshots:   0 total
Time:        508.857s
Ran all test suites.
A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.
✨  Done in 518.61s.

Second time yarn test failed (didn't check for existing parity or bitcoind nodes):

Details

yarn test
yarn run v1.22.0
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 3.19s
$ jest
 FAIL  tests/bitcoin_ethereum.ts
  ● Test suite failed to run

    Lock file is already being held

      at node_modules/proper-lockfile/lib/lockfile.js:68:47
      at callback (node_modules/graceful-fs/polyfills.js:295:20)

 FAIL  tests/ethereum_bitcoin.ts
  ● Test suite failed to run

    Lock file is already being held

      at node_modules/proper-lockfile/lib/lockfile.js:68:47
      at callback (node_modules/graceful-fs/polyfills.js:295:20)

 PASS  tests/lightning_routes.ts (10.246s)
(node:72340) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:72339) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 FAIL  tests/lnd_sanity.ts
  ● Test suite failed to run

    timed out after 60000ms

      51 |         const id = setTimeout(() => {
      52 |             clearTimeout(id);
    > 53 |             reject(new Error(`timed out after ${ms}ms`));
         |                    ^
      54 |         }, ms);
      55 |     });
      56 | 

      at Timeout.<anonymous> (src/utils.ts:53:20)

 PASS  tests/sanity.ts (18.07s)
 PASS  tests/rfc003_schema.ts (11.388s)
 PASS  tests/multiple_peers.ts (7.041s)
 PASS  tests/peers_using_ip.ts (8.255s)

Test Suites: 3 failed, 5 passed, 8 total
Tests:       22 passed, 22 total
Snapshots:   0 total
Time:        75.602s, estimated 470s
Ran all test suites.
A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Killed again bitcoind and parity.
Third run of yarn test: test passed but bitcoind and parity are still running after the test (this probably caused the test failure above)

Details

yarn test
yarn run v1.22.0
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 0.68s
$ jest
 PASS  tests/lightning_routes.ts (13.157s)
(node:72510) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:72511) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/rfc003_schema.ts (17.093s)
(node:72509) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/multiple_peers.ts
 PASS  tests/sanity.ts (21.293s)
 PASS  tests/peers_using_ip.ts (8.559s)
 PASS  tests/lnd_sanity.ts (49.123s)
 PASS  tests/ethereum_bitcoin.ts (200.896s)
 PASS  tests/bitcoin_ethereum.ts (456.904s)

Test Suites: 8 passed, 8 total
Tests:       44 passed, 44 total
Snapshots:   0 total
Time:        482.882s
Ran all test suites.
A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.
✨  Done in 493.00s.

Conclusion, as you mention: nodes do not shutdown after the tests, but once killed, the test run :)

[thomaseizinger]

Status from my machine:
make e2e failed:
Details (click here)

After killing parity and bitcoind the first run yarn test passed
Details (click here)

Second time yarn test failed (didn't check for existing parity or bitcoind nodes):
Details

Killed again bitcoind and parity.
Third run of yarn test: test passed but bitcoind and parity are still running after the test (this probably caused the test failure above)
Details

Conclusion, as you mention: nodes do not shutdown after the tests, but once killed, the test run :)

@bonomat I just pushed some changes. The nodes should now be killed properly, even after a successful test run and before we start up (if there are some left over nodes). Can't get it to work for failing tests though.

[luckysori]

It LGTM. I think I've reviewed some of this before :O

[luckysori]

❓ What is a docblock pragma?

[thomaseizinger]

The @foobar stuff in doc comments above the tests :)

[thomaseizinger]

It LGTM. I think I've reviewed some of this before :O

Yes there was a PR at some point but I had to follow a different approach in the end.

[da-kami]

Not sure what to do with this PR at the moment.

I ran yarn test 6 times and tracked some findings:

• ✅ Ran successfully.
• 🛑 2 ethereum_bitcoin tests timed out.
  • had to cleanup bitcoind / parity
• 🛑 2 bitcoin_ethereum tests timed out.
  • had to cleanup bitcoind / parity
• 🛑 The lnd_sanity test failed.
  • probably should have cleaned up lnd but did not
• ✅ Ran successfully.
  • checked, all cleaned up
• ✅ Ran successfully
  • checked, all cleaned up

I always checked for remaining bitcoind / parity instances after each run.

Note that I noticed that around 20 instances of lnd were actually running on my machine... (never checked for lnd before)

After cleaning the lnd instances up and running the suite again the lnd_sanity test passed.

Findings:

• It seems when it is a failed scenario only the nodes that are concerned by the fail-scenario remain up. Is that expected @thomaseizinger ? (I think yes, because the the lnd_sanity test is teared down separately from bitcoin/ethereum tests.)
  • It properly shutdown bitcoind and parity after the lnd_sanity test failed.
  • It does not properly shutdown bitcoind and parity if a bitcoin_ethereum / ethereum_bitcoin fails
• It seems the lnd_sanity test failing was related to the lnd instances running on my machine. (unfortunately never checked for that), so that "problem" might be solved for now.
• It seems when all instances are cleaned up properly chances are higher that there is no test failure. One thing that I don't understand is: Why would remining instances of lnd/bitcoind/parity lead to test failures? We run them on different ports as far as I could see... How could that have an influence?

Conclusion:

I feel our tests are generally a bit brittle. The bitcoin/ethereum failures seems to be related to time again somehow. I would need to further investigate where the problems actually are. (it's a bit weird, because it seems it just does not start the swap - the logs just don't print any activity after start of cnd but also no error.)

I'm quite sure this is not (just) related to this PR. Not sure if this is already approval ready. Thoughts, @thomaseizinger ?

[bonomat]

I ran it a few times again:

here are the stats:

run 1: lnd_sanity.ts failed / parity&bitcoind: running

$ jest --forceExit
 FAIL  tests/lnd_sanity.ts
  ● Test suite failed to run

    OpenStatusUpdate is malformed: {"chanPending":{"txid":{"type":"Buffer","data":[110,83,150,108,9,45,102,135,153,223,176,167,82,14,203,250,234,10,210,13,104,178,123,77,190,163,17,69,62,116,27,50]}},"pendingChanId":{"type":"Buffer","data":[182,229,8,149,143,114,223,192,62,87,246,246,90,103,85,61,74,125,183,214,235,193,34,124,29,222,24,12,73,204,8,66]}}

      at outpointFromChannelStatusUpdate (node_modules/comit-sdk/src/wallet/lightning.ts:210:9)
      at LightningWallet.<anonymous> (node_modules/comit-sdk/src/wallet/lightning.ts:110:12)
      at fulfilled (node_modules/comit-sdk/dist/src/wallet/lightning.js:5:58)

run 2: passed / parity & bitcoind: killed

run 3: passed / parity & bitcoind: killed

run 4: failed / parity & bitcoind: running:

 FAIL  tests/ethereum_bitcoin.ts (238.038s)
  ● E2E: Ethereum/ether - Bitcoin/bitcoin › rfc003-eth-btc-alice-redeems-with-high-fee

    timed out after 60000ms

      51 |         const id = setTimeout(() => {
      52 |             clearTimeout(id);
    > 53 |             reject(new Error(`timed out after ${ms}ms`));
         |                    ^
      54 |         }, ms);
      55 |     });
      56 | 

      at Timeout.<anonymous> (src/utils.ts:53:20)

At the moment I'm running dev 4 times to see what's the behavior there.

I can't say for sure but I would assume that it's parity that got suck.

Note: not approving or rejecting this PR. Let's find a solution first.

[bonomat]

what was the reason for moving it out of the e2e folder? I liked the folder structure :)

[thomaseizinger]

It just added noise IMO and our new test_environment does not longer care about dry vs e2e tests. I just runs tests and starts things if the tests require that.

I would expect that the structure of these tests gets a lot clearer with #2193.

[bonomat]

@thomaseizinger : I took over and rebased against origin/dev.

[bonomat]

I ran this branch 5 times in a row, everytime with success.

There is a known limitation: in a failure case the nodes do not terminate.

Nevertheless I opt for merging this if the happy case works for everyone and create a high priority follow up ticket to tackle the error case.

@thomaseizinger , @da-kami , @luckysori , @tcharding , @rishflab , @D4nte , can you run the e2e tests from this branch please and report the results?

you can use this command and just let it run it 5 times in a row (note, the log files will be overwritten on every run):

for i in 1 2 3 4 5
do
   echo "Test run $i"
   yarn test --verbose
done

p.s. GitHub Action seem to fail but I don't think this is related to this branch.

Test results

for i in 1 2 3 4 5
do
   echo "Test run $i"
   yarn test --verbose > log_$i.log
done
Test run 1
    Finished dev [unoptimized + debuginfo] target(s) in 0.37s
Found pid file lnd-alice/lnd.pid, sending SIGINT to process with PID 74653
Found pid file lnd-bob/lnd.pid, sending SIGINT to process with PID 74654
PASS tests/lnd_sanity.ts (67.84s)
  E2E: Sanity - LND Alice pays Bob
    ✓ sanity-lnd-alice-pays-bob (10081ms)
    ✓ sanity-lnd-alice-pays-bob-using-hold-invoice (10424ms)

(node:74836) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/sanity.ts (14.953s)
  Sanity
    ✓ invalid-swap-yields-404 (1613ms)
    ✓ empty-swap-list-after-startup (1531ms)
    ✓ bad-request-for-invalid-swap-combination (1565ms)
    ✓ returns-invalid-body-for-bad-json (1580ms)
    ✓ alice-has-empty-peer-list (1552ms)
    ✓ returns-listen-addresses-on-root-document (1528ms)
    ✓ can-fetch-root-document-as-siren (1536ms)
    ✓ returns-listen-addresses-on-root-document-as-siren (1558ms)
    ✓ returns-links-to-create-swap-endpoints-on-root-document-as-siren (1542ms)

(node:74836) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/rfc003_schema.ts (10.411s)
  Rfc003 schema tests
    ✓ get-all-swaps-is-valid-siren (1721ms)
    ✓ get-single-swap-is-valid-siren (1690ms)
    ✓ get-single-swap-contains-link-to-rfc (1630ms)
  Rfc003 schema swap reject tests
    ✓ alice-can-make-default-swap-request (1611ms)
    ✓ bob-can-decline-swap (2824ms)

PASS tests/peers_using_ip.ts (7.736s)
  Peers using IP tests
    ✓ alice-empty-peer-list (1559ms)
    ✓ alice-send-request-wrong-peer-id (2679ms)
    ✓ alice-send-swap-request-to-charlie (2628ms)

PASS tests/lightning_routes.ts (7.056s)
  Lightning routes tests
    ✓ lightning-routes-post-eth-lnbtc-return-400 (1567ms)
    ✓ lightning-routes-post-erc20-lnbtc-return-400 (1529ms)
    ✓ lightning-routes-post-lnbtc-eth-return-400 (1575ms)
    ✓ lightning-routes-post-lnbtc-erc20-return-400 (1536ms)

PASS tests/multiple_peers.ts
  Multiple peers tests
    ✓ alice-sends-swap-request-to-bob-and-charlie (1664ms)

(node:74834) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:74835) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/ethereum_bitcoin.ts (204.645s)
  E2E: Ethereum/ether - Bitcoin/bitcoin
    ✓ rfc003-eth-btc-alice-redeems-bob-redeems (33379ms)
    ✓ rfc003-eth-btc-alpha-deploy-fails (30010ms)
    ✓ rfc003-eth-btc-bob-refunds-alice-refunds (29365ms)
    ✓ rfc003-eth-btc-alice-redeems-with-high-fee (22254ms)
  E2E: Ethereum/erc20 - Bitcoin/bitcoin
    ✓ rfc003-eth-erc20_btc-alice-redeems-bob-redeems (28050ms)
    ✓ rfc003-eth-erc20_btc-bob-refunds-alice-refunds (31166ms)

PASS tests/bitcoin_ethereum.ts (429.383s)
  E2E: Bitcoin/bitcoin - Ethereum/ether
    ✓ rfc003-btc-eth-alice-redeems-bob-redeems (29955ms)
    ✓ rfc003-btc-eth-bob-refunds-alice-refunds (30732ms)
    ✓ rfc003-btc-eth-alice-refunds-bob-refunds (28123ms)
    ✓ rfc003-btc-eth-cnd-can-be-restarted (15075ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-funds (25898ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-redeems (28781ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-funds (31333ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-redeems (21272ms)
    ✓ rfc003-btc-eth-alice-underfunds-bob-aborts (34234ms)
    ✓ rfc003-btc-eth-bob-underfunds-both-refund (30487ms)
    ✓ rfc003-btc-eth-alice-overfunds-bob-aborts (33971ms)
    ✓ rfc003-btc-eth-bob-overfunds-both-refund (29984ms)
  E2E: Bitcoin/bitcoin - Ethereum/erc20
    ✓ rfc003-btc-eth-erc20-alice-redeems-bob-redeems (26442ms)
    ✓ rfc003-btc-eth-erc20-bob-refunds-alice-refunds (27653ms)

Test Suites: 8 passed, 8 total
Tests:       44 passed, 44 total
Snapshots:   0 total
Time:        432.915s, estimated 454s
Ran all test suites.
Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 74841
Found pid file lnd-alice/lnd.pid, sending SIGINT to process with PID 74845
Found pid file lnd-bob/lnd.pid, sending SIGINT to process with PID 74846
Found pid file parity/parity.pid, sending SIGINT to process with PID 74840
Found pid file parity/parity.pid, sending SIGINT to process with PID 74840
Error: kill ESRCH
    at process.kill (internal/process/per_thread.js:196:13)
    at Object.<anonymous> (/Users/bonomat/src/github/comit/comit-rs/api_tests/src/environment/kill_nodes.ts:33:25)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/bonomat/src/github/comit/comit-rs/api_tests/src/environment/kill_nodes.ts:5:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
error Command failed with exit code 1.
Test run 2
    Finished dev [unoptimized + debuginfo] target(s) in 0.96s
PASS tests/lnd_sanity.ts (56.676s)
  E2E: Sanity - LND Alice pays Bob
    ✓ sanity-lnd-alice-pays-bob (9572ms)
    ✓ sanity-lnd-alice-pays-bob-using-hold-invoice (9983ms)

(node:75047) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/sanity.ts (14.777s)
  Sanity
    ✓ invalid-swap-yields-404 (1549ms)
    ✓ empty-swap-list-after-startup (1530ms)
    ✓ bad-request-for-invalid-swap-combination (1533ms)
    ✓ returns-invalid-body-for-bad-json (1531ms)
    ✓ alice-has-empty-peer-list (1547ms)
    ✓ returns-listen-addresses-on-root-document (1555ms)
    ✓ can-fetch-root-document-as-siren (1536ms)
    ✓ returns-listen-addresses-on-root-document-as-siren (1545ms)
    ✓ returns-links-to-create-swap-endpoints-on-root-document-as-siren (1537ms)

(node:75047) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/rfc003_schema.ts (9.91s)
  Rfc003 schema tests
    ✓ get-all-swaps-is-valid-siren (1563ms)
    ✓ get-single-swap-is-valid-siren (1609ms)
    ✓ get-single-swap-contains-link-to-rfc (1594ms)
  Rfc003 schema swap reject tests
    ✓ alice-can-make-default-swap-request (1723ms)
    ✓ bob-can-decline-swap (2712ms)

PASS tests/peers_using_ip.ts (7.703s)
  Peers using IP tests
    ✓ alice-empty-peer-list (1725ms)
    ✓ alice-send-request-wrong-peer-id (2613ms)
    ✓ alice-send-swap-request-to-charlie (2638ms)

PASS tests/lightning_routes.ts (7.432s)
  Lightning routes tests
    ✓ lightning-routes-post-eth-lnbtc-return-400 (1603ms)
    ✓ lightning-routes-post-erc20-lnbtc-return-400 (1571ms)
    ✓ lightning-routes-post-lnbtc-eth-return-400 (1536ms)
    ✓ lightning-routes-post-lnbtc-erc20-return-400 (1535ms)

PASS tests/multiple_peers.ts
  Multiple peers tests
    ✓ alice-sends-swap-request-to-bob-and-charlie (1945ms)

(node:75045) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:75046) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/ethereum_bitcoin.ts (193.106s)
  E2E: Ethereum/ether - Bitcoin/bitcoin
    ✓ rfc003-eth-btc-alice-redeems-bob-redeems (24380ms)
    ✓ rfc003-eth-btc-alpha-deploy-fails (29231ms)
    ✓ rfc003-eth-btc-bob-refunds-alice-refunds (24903ms)
    ✓ rfc003-eth-btc-alice-redeems-with-high-fee (21077ms)
  E2E: Ethereum/erc20 - Bitcoin/bitcoin
    ✓ rfc003-eth-erc20_btc-alice-redeems-bob-redeems (29080ms)
    ✓ rfc003-eth-erc20_btc-bob-refunds-alice-refunds (30280ms)

PASS tests/bitcoin_ethereum.ts (418.606s)
  E2E: Bitcoin/bitcoin - Ethereum/ether
    ✓ rfc003-btc-eth-alice-redeems-bob-redeems (30038ms)
    ✓ rfc003-btc-eth-bob-refunds-alice-refunds (30539ms)
    ✓ rfc003-btc-eth-alice-refunds-bob-refunds (28832ms)
    ✓ rfc003-btc-eth-cnd-can-be-restarted (9482ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-funds (31254ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-redeems (23991ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-funds (30420ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-redeems (25809ms)
    ✓ rfc003-btc-eth-alice-underfunds-bob-aborts (33349ms)
    ✓ rfc003-btc-eth-bob-underfunds-both-refund (25423ms)
    ✓ rfc003-btc-eth-alice-overfunds-bob-aborts (33082ms)
    ✓ rfc003-btc-eth-bob-overfunds-both-refund (30637ms)
  E2E: Bitcoin/bitcoin - Ethereum/erc20
    ✓ rfc003-btc-eth-erc20-alice-redeems-bob-redeems (26272ms)
    ✓ rfc003-btc-eth-erc20-bob-refunds-alice-refunds (33270ms)

Test Suites: 8 passed, 8 total
Tests:       44 passed, 44 total
Snapshots:   0 total
Time:        421.822s, estimated 430s
Ran all test suites.
Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 75051
Found pid file lnd-alice/lnd.pid, sending SIGINT to process with PID 75057
Found pid file lnd-bob/lnd.pid, sending SIGINT to process with PID 75058
Found pid file parity/parity.pid, sending SIGINT to process with PID 75052
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?
Test run 3
    Finished dev [unoptimized + debuginfo] target(s) in 0.89s
PASS tests/lnd_sanity.ts (57.084s)
  E2E: Sanity - LND Alice pays Bob
    ✓ sanity-lnd-alice-pays-bob (5920ms)
    ✓ sanity-lnd-alice-pays-bob-using-hold-invoice (5181ms)

(node:75253) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/sanity.ts (14.609s)
  Sanity
    ✓ invalid-swap-yields-404 (1549ms)
    ✓ empty-swap-list-after-startup (1558ms)
    ✓ bad-request-for-invalid-swap-combination (1568ms)
    ✓ returns-invalid-body-for-bad-json (1551ms)
    ✓ alice-has-empty-peer-list (1531ms)
    ✓ returns-listen-addresses-on-root-document (1548ms)
    ✓ can-fetch-root-document-as-siren (1539ms)
    ✓ returns-listen-addresses-on-root-document-as-siren (1536ms)
    ✓ returns-links-to-create-swap-endpoints-on-root-document-as-siren (1547ms)

(node:75252) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:75251) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:75253) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
FAIL tests/rfc003_schema.ts (130.993s)
  Rfc003 schema tests
    ✓ get-all-swaps-is-valid-siren (121794ms)
    ✓ get-single-swap-is-valid-siren (2287ms)
    ✓ get-single-swap-contains-link-to-rfc (1573ms)
  Rfc003 schema swap reject tests
    ✓ alice-can-make-default-swap-request (1860ms)
    ✓ bob-can-decline-swap (2668ms)

Test Suites: 8 passed, 8 total
Tests:       44 passed, 44 total
Snapshots:   0 total
Time:        441.794s
Ran all test suites.
    Finished dev [unoptimized + debuginfo] target(s) in 2.13s
Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 75258
Found pid file lnd-alice/lnd.pid, sending SIGINT to process with PID 75266
Found pid file lnd-bob/lnd.pid, sending SIGINT to process with PID 75267
Found pid file parity/parity.pid, sending SIGINT to process with PID 75257
PASS tests/lnd_sanity.ts (49.137s)
  E2E: Sanity - LND Alice pays Bob
    ✓ sanity-lnd-alice-pays-bob (4796ms)
    ✓ sanity-lnd-alice-pays-bob-using-hold-invoice (5245ms)

(node:75433) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/sanity.ts (19.619s)
  Sanity
    ✓ invalid-swap-yields-404 (1541ms)
    ✓ empty-swap-list-after-startup (1536ms)
    ✓ bad-request-for-invalid-swap-combination (1551ms)
    ✓ returns-invalid-body-for-bad-json (1537ms)
    ✓ alice-has-empty-peer-list (6559ms)
    ✓ returns-listen-addresses-on-root-document (1551ms)
    ✓ can-fetch-root-document-as-siren (1534ms)
    ✓ returns-listen-addresses-on-root-document-as-siren (1540ms)
    ✓ returns-links-to-create-swap-endpoints-on-root-document-as-siren (1537ms)

(node:75433) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/rfc003_schema.ts (10.238s)
  Rfc003 schema tests
    ✓ get-all-swaps-is-valid-siren (1548ms)
    ✓ get-single-swap-is-valid-siren (2128ms)
    ✓ get-single-swap-contains-link-to-rfc (1666ms)
  Rfc003 schema swap reject tests
    ✓ alice-can-make-default-swap-request (1619ms)
    ✓ bob-can-decline-swap (2622ms)

PASS tests/peers_using_ip.ts (8.653s)
  Peers using IP tests
    ✓ alice-empty-peer-list (1794ms)
    ✓ alice-send-request-wrong-peer-id (2658ms)
    ✓ alice-send-swap-request-to-charlie (2606ms)

PASS tests/lightning_routes.ts (7.121s)
  Lightning routes tests
    ✓ lightning-routes-post-eth-lnbtc-return-400 (1562ms)
    ✓ lightning-routes-post-erc20-lnbtc-return-400 (1552ms)
    ✓ lightning-routes-post-lnbtc-eth-return-400 (1603ms)
    ✓ lightning-routes-post-lnbtc-erc20-return-400 (1553ms)

PASS tests/multiple_peers.ts
  Multiple peers tests
    ✓ alice-sends-swap-request-to-bob-and-charlie (1674ms)

(node:75432) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:75431) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/ethereum_bitcoin.ts (178.656s)
  E2E: Ethereum/ether - Bitcoin/bitcoin
    ✓ rfc003-eth-btc-alice-redeems-bob-redeems (27696ms)
    ✓ rfc003-eth-btc-alpha-deploy-fails (24733ms)
    ✓ rfc003-eth-btc-bob-refunds-alice-refunds (23157ms)
    ✓ rfc003-eth-btc-alice-redeems-with-high-fee (16482ms)
  E2E: Ethereum/erc20 - Bitcoin/bitcoin
    ✓ rfc003-eth-erc20_btc-alice-redeems-bob-redeems (27324ms)
    ✓ rfc003-eth-erc20_btc-bob-refunds-alice-refunds (31699ms)

PASS tests/bitcoin_ethereum.ts (418.676s)
  E2E: Bitcoin/bitcoin - Ethereum/ether
    ✓ rfc003-btc-eth-alice-redeems-bob-redeems (23591ms)
    ✓ rfc003-btc-eth-bob-refunds-alice-refunds (31494ms)
    ✓ rfc003-btc-eth-alice-refunds-bob-refunds (28010ms)
    ✓ rfc003-btc-eth-cnd-can-be-restarted (14263ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-funds (30560ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-redeems (24860ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-funds (31000ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-redeems (25277ms)
    ✓ rfc003-btc-eth-alice-underfunds-bob-aborts (28814ms)
    ✓ rfc003-btc-eth-bob-underfunds-both-refund (25157ms)
    ✓ rfc003-btc-eth-alice-overfunds-bob-aborts (33516ms)
    ✓ rfc003-btc-eth-bob-overfunds-both-refund (30895ms)
  E2E: Bitcoin/bitcoin - Ethereum/erc20
    ✓ rfc003-btc-eth-erc20-alice-redeems-bob-redeems (30448ms)
    ✓ rfc003-btc-eth-erc20-bob-refunds-alice-refunds (27451ms)

Test Suites: 8 passed, 8 total
Tests:       44 passed, 44 total
Snapshots:   0 total
Time:        421.797s
Ran all test suites.
Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 75438
Found pid file lnd-alice/lnd.pid, sending SIGINT to process with PID 75443
Found pid file lnd-bob/lnd.pid, sending SIGINT to process with PID 75444
Found pid file parity/parity.pid, sending SIGINT to process with PID 75437
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?
Test run 5
    Finished dev [unoptimized + debuginfo] target(s) in 0.89s
PASS tests/lnd_sanity.ts (44.481s)
  E2E: Sanity - LND Alice pays Bob
    ✓ sanity-lnd-alice-pays-bob (4538ms)
    ✓ sanity-lnd-alice-pays-bob-using-hold-invoice (4982ms)

(node:75643) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/sanity.ts (14.79s)
  Sanity
    ✓ invalid-swap-yields-404 (1545ms)
    ✓ empty-swap-list-after-startup (1564ms)
    ✓ bad-request-for-invalid-swap-combination (1554ms)
    ✓ returns-invalid-body-for-bad-json (1548ms)
    ✓ alice-has-empty-peer-list (1534ms)
    ✓ returns-listen-addresses-on-root-document (1532ms)
    ✓ can-fetch-root-document-as-siren (1538ms)
    ✓ returns-listen-addresses-on-root-document-as-siren (1526ms)
    ✓ returns-links-to-create-swap-endpoints-on-root-document-as-siren (1558ms)

(node:75643) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/rfc003_schema.ts (10.072s)
  Rfc003 schema tests
    ✓ get-all-swaps-is-valid-siren (1644ms)
    ✓ get-single-swap-is-valid-siren (1607ms)
    ✓ get-single-swap-contains-link-to-rfc (1577ms)
  Rfc003 schema swap reject tests
    ✓ alice-can-make-default-swap-request (1684ms)
    ✓ bob-can-decline-swap (2664ms)

PASS tests/peers_using_ip.ts (7.907s)
  Peers using IP tests
    ✓ alice-empty-peer-list (1682ms)
    ✓ alice-send-request-wrong-peer-id (2755ms)
    ✓ alice-send-swap-request-to-charlie (2658ms)

PASS tests/lightning_routes.ts (7.044s)
  Lightning routes tests
    ✓ lightning-routes-post-eth-lnbtc-return-400 (1583ms)
    ✓ lightning-routes-post-erc20-lnbtc-return-400 (1549ms)
    ✓ lightning-routes-post-lnbtc-eth-return-400 (1533ms)
    ✓ lightning-routes-post-lnbtc-erc20-return-400 (1534ms)

PASS tests/multiple_peers.ts
  Multiple peers tests
    ✓ alice-sends-swap-request-to-bob-and-charlie (2167ms)

(node:75641) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:75642) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
PASS tests/ethereum_bitcoin.ts (194.992s)
  E2E: Ethereum/ether - Bitcoin/bitcoin
    ✓ rfc003-eth-btc-alice-redeems-bob-redeems (24731ms)
    ✓ rfc003-eth-btc-alpha-deploy-fails (28902ms)
    ✓ rfc003-eth-btc-bob-refunds-alice-refunds (24071ms)
    ✓ rfc003-eth-btc-alice-redeems-with-high-fee (21309ms)
  E2E: Ethereum/erc20 - Bitcoin/bitcoin
    ✓ rfc003-eth-erc20_btc-alice-redeems-bob-redeems (27916ms)
    ✓ rfc003-eth-erc20_btc-bob-refunds-alice-refunds (35000ms)

PASS tests/bitcoin_ethereum.ts (428.398s)
  E2E: Bitcoin/bitcoin - Ethereum/ether
    ✓ rfc003-btc-eth-alice-redeems-bob-redeems (28881ms)
    ✓ rfc003-btc-eth-bob-refunds-alice-refunds (30342ms)
    ✓ rfc003-btc-eth-alice-refunds-bob-refunds (29885ms)
    ✓ rfc003-btc-eth-cnd-can-be-restarted (10116ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-funds (30733ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-redeems (24821ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-funds (25518ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-redeems (30600ms)
    ✓ rfc003-btc-eth-alice-underfunds-bob-aborts (33569ms)
    ✓ rfc003-btc-eth-bob-underfunds-both-refund (25510ms)
    ✓ rfc003-btc-eth-alice-overfunds-bob-aborts (33372ms)
    ✓ rfc003-btc-eth-bob-overfunds-both-refund (30352ms)
  E2E: Bitcoin/bitcoin - Ethereum/erc20
    ✓ rfc003-btc-eth-erc20-alice-redeems-bob-redeems (35520ms)
    ✓ rfc003-btc-eth-erc20-bob-refunds-alice-refunds (33327ms)

Test Suites: 8 passed, 8 total
Tests:       44 passed, 44 total
Snapshots:   0 total
Time:        431.794s
Ran all test suites.
Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 75648
Found pid file lnd-alice/lnd.pid, sending SIGINT to process with PID 75652
Found pid file lnd-bob/lnd.pid, sending SIGINT to process with PID 75653
Found pid file parity/parity.pid, sending SIGINT to process with PID 75647
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?

[luckysori]

Ignore the Lightning tests because I forgot to add lnd to the path.

I see some things that concern me:

1. Some tests time out.
2. This warning is popping up again despite having been eliminated (AFAIK) with Close all handles during Jest teardown #2330:

(node:22187) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit

3. Signs of improper teardown, which was also supposed to be fixed with Close all handles during Jest teardown #2330:

A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.

Test results

➜  api_tests git:(f3a477f6) for i in 1 2 3 4 5
do
   echo "Test run $i"
   yarn test --verbose
done
Test run 1
yarn run v1.21.1
$ cargo build --bin cnd && tsc
  Downloaded paste v0.1.9
  Downloaded thiserror v1.0.14
  Downloaded derivative v2.1.0
  Downloaded paste-impl v0.1.9
  Downloaded thiserror-impl v1.0.14
   Compiling cnd v0.7.2 (/home/luckysori/work/comit-network/comit-rs/cnd)
   Compiling thiserror-impl v1.0.14
   Compiling derivative v2.1.0
   Compiling paste-impl v0.1.9
   Compiling serde_json v1.0.50
   Compiling paste v0.1.9
   Compiling wasm-bindgen v0.2.58
   Compiling warp v0.2.2
   Compiling siren-types v0.2.0
   Compiling reqwest v0.10.4
   Compiling blockchain_contracts v0.3.1
   Compiling tracing-subscriber v0.2.3
   Compiling thiserror v1.0.14
   Compiling js-sys v0.3.35
   Compiling libp2p-core v0.16.0
   Compiling yamux v0.4.3
   Compiling http-api-problem v0.15.0
   Compiling libp2p-swarm v0.16.1
   Compiling libp2p-uds v0.16.0
   Compiling libp2p-secio v0.16.1
   Compiling libp2p-mplex v0.16.0
   Compiling libp2p-dns v0.16.0
   Compiling libp2p-plaintext v0.16.0
   Compiling libp2p-tcp v0.16.0
   Compiling libp2p-deflate v0.16.0
   Compiling libp2p-noise v0.16.2
   Compiling libp2p-yamux v0.16.2
   Compiling wasm-bindgen-futures v0.4.8
   Compiling libp2p-mdns v0.16.0
   Compiling libp2p-floodsub v0.16.0
   Compiling libp2p-kad v0.16.2
   Compiling libp2p-ping v0.16.0
   Compiling libp2p-identify v0.16.0
   Compiling libp2p-gossipsub v0.16.0
   Compiling libp2p-wasm-ext v0.16.2
   Compiling libp2p v0.16.2
   Compiling libp2p-comit v0.1.0 (/home/luckysori/work/comit-network/comit-rs/libp2p-comit)
    Finished dev [unoptimized + debuginfo] target(s) in 3m 11s
$ jest --forceExit --verbose
 PASS  tests/multiple_peers.ts (19.38s)
  Multiple peers tests
    ✓ alice-sends-swap-request-to-bob-and-charlie (1906ms)

 FAIL  tests/lnd_sanity.ts
  ● Test suite failed to run

    Could not find lnd on your system; Command failed: which lnd

 PASS  tests/lightning_routes.ts (23.378s)
  Lightning routes tests
    ✓ lightning-routes-post-eth-lnbtc-return-400 (1614ms)
    ✓ lightning-routes-post-erc20-lnbtc-return-400 (1554ms)
    ✓ lightning-routes-post-lnbtc-eth-return-400 (1541ms)
    ✓ lightning-routes-post-lnbtc-erc20-return-400 (1529ms)

 PASS  tests/peers_using_ip.ts (24.65s)
  Peers using IP tests
    ✓ alice-empty-peer-list (1619ms)
    ✓ alice-send-request-wrong-peer-id (2723ms)
    ✓ alice-send-swap-request-to-charlie (2670ms)

(node:22167) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/rfc003_schema.ts (28.352s)
  Rfc003 schema tests
    ✓ get-all-swaps-is-valid-siren (1819ms)
    ✓ get-single-swap-is-valid-siren (2194ms)
    ✓ get-single-swap-contains-link-to-rfc (1605ms)
  Rfc003 schema swap reject tests
    ✓ alice-can-make-default-swap-request (1723ms)
    ✓ bob-can-decline-swap (2748ms)

(node:22174) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/sanity.ts (32.084s)
  Sanity
    ✓ invalid-swap-yields-404 (1666ms)
    ✓ empty-swap-list-after-startup (1552ms)
    ✓ bad-request-for-invalid-swap-combination (1531ms)
    ✓ returns-invalid-body-for-bad-json (1531ms)
    ✓ alice-has-empty-peer-list (1531ms)
    ✓ returns-listen-addresses-on-root-document (1541ms)
    ✓ can-fetch-root-document-as-siren (1543ms)
    ✓ returns-listen-addresses-on-root-document-as-siren (1523ms)
    ✓ returns-links-to-create-swap-endpoints-on-root-document-as-siren (1528ms)

(node:22187) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:22160) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/ethereum_bitcoin.ts (165.824s)
  E2E: Ethereum/ether - Bitcoin/bitcoin
    ✓ rfc003-eth-btc-alice-redeems-bob-redeems (23935ms)
    ✓ rfc003-eth-btc-alpha-deploy-fails (20721ms)
    ✓ rfc003-eth-btc-bob-refunds-alice-refunds (23799ms)
    ✓ rfc003-eth-btc-alice-redeems-with-high-fee (16366ms)
  E2E: Ethereum/erc20 - Bitcoin/bitcoin
    ✓ rfc003-eth-erc20_btc-alice-redeems-bob-redeems (28583ms)
    ✓ rfc003-eth-erc20_btc-bob-refunds-alice-refunds (30431ms)

 PASS  tests/bitcoin_ethereum.ts (418.742s)
  E2E: Bitcoin/bitcoin - Ethereum/ether
    ✓ rfc003-btc-eth-alice-redeems-bob-redeems (26809ms)
    ✓ rfc003-btc-eth-bob-refunds-alice-refunds (30753ms)
    ✓ rfc003-btc-eth-alice-refunds-bob-refunds (28306ms)
    ✓ rfc003-btc-eth-cnd-can-be-restarted (10305ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-funds (30867ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-redeems (29041ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-funds (30657ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-redeems (31012ms)
    ✓ rfc003-btc-eth-alice-underfunds-bob-aborts (28428ms)
    ✓ rfc003-btc-eth-bob-underfunds-both-refund (26018ms)
    ✓ rfc003-btc-eth-alice-overfunds-bob-aborts (33619ms)
    ✓ rfc003-btc-eth-bob-overfunds-both-refund (30442ms)
  E2E: Bitcoin/bitcoin - Ethereum/erc20
    ✓ rfc003-btc-eth-erc20-alice-redeems-bob-redeems (25807ms)
    ✓ rfc003-btc-eth-erc20-bob-refunds-alice-refunds (33362ms)

Test Suites: 1 failed, 7 passed, 8 total
Tests:       42 passed, 42 total
Snapshots:   0 total
Time:        421.817s
Ran all test suites.
Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 22257
A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.
Found pid file bitcoind/miner.pid, sending SIGINT to process with PID 22375
Found pid file parity/parity.pid, sending SIGINT to process with PID 22256
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Test run 2
yarn run v1.21.1
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 0.24s
$ jest --forceExit --verbose
 PASS  tests/multiple_peers.ts (22.32s)
  Multiple peers tests
    ✓ alice-sends-swap-request-to-bob-and-charlie (2873ms)

 FAIL  tests/lnd_sanity.ts
  ● Test suite failed to run

    Could not find lnd on your system; Command failed: which lnd

 PASS  tests/lightning_routes.ts (25.212s)
  Lightning routes tests
    ✓ lightning-routes-post-eth-lnbtc-return-400 (1645ms)
    ✓ lightning-routes-post-erc20-lnbtc-return-400 (1550ms)
    ✓ lightning-routes-post-lnbtc-eth-return-400 (1553ms)
    ✓ lightning-routes-post-lnbtc-erc20-return-400 (1532ms)

(node:23412) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/peers_using_ip.ts (27.095s)
  Peers using IP tests
    ✓ alice-empty-peer-list (1610ms)
    ✓ alice-send-request-wrong-peer-id (2740ms)
    ✓ alice-send-swap-request-to-charlie (2677ms)

 PASS  tests/rfc003_schema.ts (29.145s)
  Rfc003 schema tests
    ✓ get-all-swaps-is-valid-siren (1674ms)
    ✓ get-single-swap-is-valid-siren (1801ms)
    ✓ get-single-swap-contains-link-to-rfc (1675ms)
  Rfc003 schema swap reject tests
    ✓ alice-can-make-default-swap-request (2741ms)
    ✓ bob-can-decline-swap (2760ms)

(node:23406) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/sanity.ts (33.174s)
  Sanity
    ✓ invalid-swap-yields-404 (1607ms)
    ✓ empty-swap-list-after-startup (1541ms)
    ✓ bad-request-for-invalid-swap-combination (1539ms)
    ✓ returns-invalid-body-for-bad-json (1535ms)
    ✓ alice-has-empty-peer-list (1531ms)
    ✓ returns-listen-addresses-on-root-document (1532ms)
    ✓ can-fetch-root-document-as-siren (1536ms)
    ✓ returns-listen-addresses-on-root-document-as-siren (1524ms)
    ✓ returns-links-to-create-swap-endpoints-on-root-document-as-siren (1528ms)

(node:23392) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:23399) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 FAIL  tests/ethereum_bitcoin.ts (264.626s)
  E2E: Ethereum/ether - Bitcoin/bitcoin
    ✓ rfc003-eth-btc-alice-redeems-bob-redeems (29979ms)
    ✓ rfc003-eth-btc-alpha-deploy-fails (25078ms)
    ✕ rfc003-eth-btc-bob-refunds-alice-refunds (123002ms)
    ✕ rfc003-eth-btc-alice-redeems-with-high-fee (1229ms)
  E2E: Ethereum/erc20 - Bitcoin/bitcoin
    ✓ rfc003-eth-erc20_btc-alice-redeems-bob-redeems (31917ms)
    ✓ rfc003-eth-erc20_btc-bob-refunds-alice-refunds (31824ms)

  ● E2E: Ethereum/ether - Bitcoin/bitcoin › rfc003-eth-btc-bob-refunds-alice-refunds

    : Timeout - Async callback was not invoked within the 123000ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 123000ms timeout specified by jest.setTimeout.Error:

      53 |     // ************************ //
      54 | 
    > 55 |     it(
         |     ^
      56 |         "rfc003-eth-btc-bob-refunds-alice-refunds",
      57 |         twoActorTest(async ({ alice, bob }) => {
      58 |             await alice.sendRequest(AssetKind.Ether, AssetKind.Bitcoin);

      at new Spec (node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
      at Suite.<anonymous> (tests/ethereum_bitcoin.ts:55:5)

  ● E2E: Ethereum/ether - Bitcoin/bitcoin › rfc003-eth-btc-alice-redeems-with-high-fee

    TimeoutError: Promise timed out after 120000 milliseconds

      at Timeout._onTimeout (node_modules/p-timeout/index.js:34:63)

Test Suites: 2 failed, 5 passed, 7 of 8 total
Tests:       2 failed, 26 passed, 28 total
Snapshots:   0 total
Time:        266.921s, estimated 419s
Ran all test suites.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Test run 3
yarn run v1.21.1
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 0.25s
$ jest --forceExit --verbose
Determining test suites to run...Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 23490
Found pid file bitcoind/miner.pid, sending SIGINT to process with PID 23607
Found pid file parity/parity.pid, sending SIGINT to process with PID 23481
 PASS  tests/multiple_peers.ts (21.415s)
  Multiple peers tests
    ✓ alice-sends-swap-request-to-bob-and-charlie (2876ms)

 FAIL  tests/lnd_sanity.ts
  ● Test suite failed to run

    Could not find lnd on your system; Command failed: which lnd

 PASS  tests/lightning_routes.ts (24.494s)
  Lightning routes tests
    ✓ lightning-routes-post-eth-lnbtc-return-400 (1673ms)
    ✓ lightning-routes-post-erc20-lnbtc-return-400 (1542ms)
    ✓ lightning-routes-post-lnbtc-eth-return-400 (1567ms)
    ✓ lightning-routes-post-lnbtc-erc20-return-400 (1572ms)

(node:24512) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/peers_using_ip.ts (24.716s)
  Peers using IP tests
    ✓ alice-empty-peer-list (1628ms)
    ✓ alice-send-request-wrong-peer-id (2714ms)
    ✓ alice-send-swap-request-to-charlie (2685ms)

 PASS  tests/rfc003_schema.ts (27.751s)
  Rfc003 schema tests
    ✓ get-all-swaps-is-valid-siren (1697ms)
    ✓ get-single-swap-is-valid-siren (1712ms)
    ✓ get-single-swap-contains-link-to-rfc (1618ms)
  Rfc003 schema swap reject tests
    ✓ alice-can-make-default-swap-request (1770ms)
    ✓ bob-can-decline-swap (2801ms)

(node:24505) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/sanity.ts (31.515s)
  Sanity
    ✓ invalid-swap-yields-404 (1604ms)
    ✓ empty-swap-list-after-startup (1541ms)
    ✓ bad-request-for-invalid-swap-combination (1592ms)
    ✓ returns-invalid-body-for-bad-json (1537ms)
    ✓ alice-has-empty-peer-list (1539ms)
    ✓ returns-listen-addresses-on-root-document (1535ms)
    ✓ can-fetch-root-document-as-siren (1526ms)
    ✓ returns-listen-addresses-on-root-document-as-siren (1521ms)
    ✓ returns-links-to-create-swap-endpoints-on-root-document-as-siren (1528ms)

(node:24499) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:24492) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/ethereum_bitcoin.ts (181.306s)
  E2E: Ethereum/ether - Bitcoin/bitcoin
    ✓ rfc003-eth-btc-alice-redeems-bob-redeems (24170ms)
    ✓ rfc003-eth-btc-alpha-deploy-fails (24839ms)
    ✓ rfc003-eth-btc-bob-refunds-alice-refunds (23959ms)
    ✓ rfc003-eth-btc-alice-redeems-with-high-fee (16450ms)
  E2E: Ethereum/erc20 - Bitcoin/bitcoin
    ✓ rfc003-eth-erc20_btc-alice-redeems-bob-redeems (34219ms)
    ✓ rfc003-eth-erc20_btc-bob-refunds-alice-refunds (35949ms)

 FAIL  tests/bitcoin_ethereum.ts (478.959s)
  E2E: Bitcoin/bitcoin - Ethereum/ether
    ✓ rfc003-btc-eth-alice-redeems-bob-redeems (27699ms)
    ✓ rfc003-btc-eth-bob-refunds-alice-refunds (30508ms)
    ✓ rfc003-btc-eth-alice-refunds-bob-refunds (28702ms)
    ✓ rfc003-btc-eth-cnd-can-be-restarted (10092ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-funds (25672ms)
    ✕ rfc003-btc-eth-resume-alice-down-bob-redeems (123002ms)
    ✕ rfc003-btc-eth-resume-bob-down-alice-funds (1053ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-redeems (33693ms)
    ✓ rfc003-btc-eth-alice-underfunds-bob-aborts (28531ms)
    ✓ rfc003-btc-eth-bob-underfunds-both-refund (30598ms)
    ✓ rfc003-btc-eth-alice-overfunds-bob-aborts (33607ms)
    ✓ rfc003-btc-eth-bob-overfunds-both-refund (30458ms)
  E2E: Bitcoin/bitcoin - Ethereum/erc20
    ✓ rfc003-btc-eth-erc20-alice-redeems-bob-redeems (26385ms)
    ✓ rfc003-btc-eth-erc20-bob-refunds-alice-refunds (27906ms)

  ● E2E: Bitcoin/bitcoin - Ethereum/ether › rfc003-btc-eth-resume-alice-down-bob-redeems

    : Timeout - Async callback was not invoked within the 123000ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 123000ms timeout specified by jest.setTimeout.Error:

      118 |     );
      119 | 
    > 120 |     it(
          |     ^
      121 |         "rfc003-btc-eth-resume-alice-down-bob-redeems",
      122 |         twoActorTest(async ({ alice, bob }) => {
      123 |             await alice.sendRequest(AssetKind.Bitcoin, AssetKind.Ether);

      at new Spec (node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
      at Suite.<anonymous> (tests/bitcoin_ethereum.ts:120:5)

  ● E2E: Bitcoin/bitcoin - Ethereum/ether › rfc003-btc-eth-resume-bob-down-alice-funds

    TimeoutError: Promise timed out after 120000 milliseconds

      at Timeout._onTimeout (node_modules/p-timeout/index.js:34:63)

Test Suites: 2 failed, 6 passed, 8 total
Tests:       2 failed, 40 passed, 42 total
Snapshots:   0 total
Time:        481.636s
Ran all test suites.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Test run 4
yarn run v1.21.1
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 0.26s
$ jest --forceExit --verbose
Determining test suites to run...Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 24578
Found pid file parity/parity.pid, sending SIGINT to process with PID 24577
 PASS  tests/multiple_peers.ts (19.437s)
  Multiple peers tests
    ✓ alice-sends-swap-request-to-bob-and-charlie (2895ms)

 FAIL  tests/lnd_sanity.ts
  ● Test suite failed to run

    Could not find lnd on your system; Command failed: which lnd

 PASS  tests/lightning_routes.ts (22.379s)
  Lightning routes tests
    ✓ lightning-routes-post-eth-lnbtc-return-400 (1628ms)
    ✓ lightning-routes-post-erc20-lnbtc-return-400 (1544ms)
    ✓ lightning-routes-post-lnbtc-eth-return-400 (1544ms)
    ✓ lightning-routes-post-lnbtc-erc20-return-400 (1538ms)

 PASS  tests/peers_using_ip.ts (23.258s)
  Peers using IP tests
    ✓ alice-empty-peer-list (1638ms)
    ✓ alice-send-request-wrong-peer-id (2695ms)
    ✓ alice-send-swap-request-to-charlie (2671ms)

(node:25805) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/rfc003_schema.ts (26.313s)
  Rfc003 schema tests
    ✓ get-all-swaps-is-valid-siren (1647ms)
    ✓ get-single-swap-is-valid-siren (1707ms)
    ✓ get-single-swap-contains-link-to-rfc (1610ms)
  Rfc003 schema swap reject tests
    ✓ alice-can-make-default-swap-request (1724ms)
    ✓ bob-can-decline-swap (2740ms)

(node:25799) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/sanity.ts (30.733s)
  Sanity
    ✓ invalid-swap-yields-404 (1633ms)
    ✓ empty-swap-list-after-startup (1542ms)
    ✓ bad-request-for-invalid-swap-combination (1544ms)
    ✓ returns-invalid-body-for-bad-json (1530ms)
    ✓ alice-has-empty-peer-list (1531ms)
    ✓ returns-listen-addresses-on-root-document (1534ms)
    ✓ can-fetch-root-document-as-siren (1537ms)
    ✓ returns-listen-addresses-on-root-document-as-siren (1526ms)
    ✓ returns-links-to-create-swap-endpoints-on-root-document-as-siren (1527ms)

(node:25785) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:25792) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 FAIL  tests/ethereum_bitcoin.ts (323.528s)
  E2E: Ethereum/ether - Bitcoin/bitcoin
    ✓ rfc003-eth-btc-alice-redeems-bob-redeems (29860ms)
    ✓ rfc003-eth-btc-alpha-deploy-fails (26072ms)
    ✕ rfc003-eth-btc-bob-refunds-alice-refunds (123002ms)
    ✕ rfc003-eth-btc-alice-redeems-with-high-fee (963ms)
  E2E: Ethereum/erc20 - Bitcoin/bitcoin
    ✓ rfc003-eth-erc20_btc-alice-redeems-bob-redeems (34861ms)
    ✕ rfc003-eth-erc20_btc-bob-refunds-alice-refunds (88019ms)

  ● E2E: Ethereum/ether - Bitcoin/bitcoin › rfc003-eth-btc-bob-refunds-alice-refunds

    : Timeout - Async callback was not invoked within the 123000ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 123000ms timeout specified by jest.setTimeout.Error:

      53 |     // ************************ //
      54 | 
    > 55 |     it(
         |     ^
      56 |         "rfc003-eth-btc-bob-refunds-alice-refunds",
      57 |         twoActorTest(async ({ alice, bob }) => {
      58 |             await alice.sendRequest(AssetKind.Ether, AssetKind.Bitcoin);

      at new Spec (node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
      at Suite.<anonymous> (tests/ethereum_bitcoin.ts:55:5)

  ● E2E: Ethereum/ether - Bitcoin/bitcoin › rfc003-eth-btc-alice-redeems-with-high-fee

    TimeoutError: Promise timed out after 120000 milliseconds

      at Timeout._onTimeout (node_modules/p-timeout/index.js:34:63)

  ● E2E: Ethereum/erc20 - Bitcoin/bitcoin › rfc003-eth-erc20_btc-bob-refunds-alice-refunds

    TimeoutError: Promise timed out after 120000 milliseconds

      at Timeout._onTimeout (node_modules/p-timeout/index.js:34:63)

Test Suites: 2 failed, 5 passed, 7 of 8 total
Tests:       3 failed, 25 passed, 28 total
Snapshots:   0 total
Time:        325.69s, estimated 419s
Ran all test suites.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Test run 5
yarn run v1.21.1
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 0.23s
$ jest --forceExit --verbose
Determining test suites to run...Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 25878
Found pid file bitcoind/miner.pid, sending SIGINT to process with PID 25999
Found pid file parity/parity.pid, sending SIGINT to process with PID 25877
 PASS  tests/multiple_peers.ts (17.4s)
  Multiple peers tests
    ✓ alice-sends-swap-request-to-bob-and-charlie (2077ms)

 FAIL  tests/lnd_sanity.ts
  ● Test suite failed to run

    Could not find lnd on your system; Command failed: which lnd

 PASS  tests/peers_using_ip.ts (22.149s)
  Peers using IP tests
    ✓ alice-empty-peer-list (1659ms)
    ✓ alice-send-request-wrong-peer-id (2734ms)
    ✓ alice-send-swap-request-to-charlie (2745ms)

 PASS  tests/lightning_routes.ts (22.442s)
  Lightning routes tests
    ✓ lightning-routes-post-eth-lnbtc-return-400 (1615ms)
    ✓ lightning-routes-post-erc20-lnbtc-return-400 (1550ms)
    ✓ lightning-routes-post-lnbtc-eth-return-400 (1543ms)
    ✓ lightning-routes-post-lnbtc-erc20-return-400 (1543ms)

(node:26977) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/rfc003_schema.ts (26.796s)
  Rfc003 schema tests
    ✓ get-all-swaps-is-valid-siren (1630ms)
    ✓ get-single-swap-is-valid-siren (2180ms)
    ✓ get-single-swap-contains-link-to-rfc (1622ms)
  Rfc003 schema swap reject tests
    ✓ alice-can-make-default-swap-request (1753ms)
    ✓ bob-can-decline-swap (2751ms)

(node:26970) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/sanity.ts (29.848s)
  Sanity
    ✓ invalid-swap-yields-404 (1647ms)
    ✓ empty-swap-list-after-startup (1539ms)
    ✓ bad-request-for-invalid-swap-combination (1538ms)
    ✓ returns-invalid-body-for-bad-json (1540ms)
    ✓ alice-has-empty-peer-list (1558ms)
    ✓ returns-listen-addresses-on-root-document (1541ms)
    ✓ can-fetch-root-document-as-siren (1531ms)
    ✓ returns-listen-addresses-on-root-document-as-siren (1526ms)
    ✓ returns-links-to-create-swap-endpoints-on-root-document-as-siren (1521ms)

(node:26964) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:26957) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
 PASS  tests/ethereum_bitcoin.ts (171.156s)
  E2E: Ethereum/ether - Bitcoin/bitcoin
    ✓ rfc003-eth-btc-alice-redeems-bob-redeems (24177ms)
    ✓ rfc003-eth-btc-alpha-deploy-fails (21160ms)
    ✓ rfc003-eth-btc-bob-refunds-alice-refunds (22784ms)
    ✓ rfc003-eth-btc-alice-redeems-with-high-fee (16796ms)
  E2E: Ethereum/erc20 - Bitcoin/bitcoin
    ✓ rfc003-eth-erc20_btc-alice-redeems-bob-redeems (29240ms)
    ✓ rfc003-eth-erc20_btc-bob-refunds-alice-refunds (36355ms)

 PASS  tests/bitcoin_ethereum.ts (420.889s)
  E2E: Bitcoin/bitcoin - Ethereum/ether
    ✓ rfc003-btc-eth-alice-redeems-bob-redeems (27138ms)
    ✓ rfc003-btc-eth-bob-refunds-alice-refunds (25899ms)
    ✓ rfc003-btc-eth-alice-refunds-bob-refunds (34237ms)
    ✓ rfc003-btc-eth-cnd-can-be-restarted (14742ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-funds (26675ms)
    ✓ rfc003-btc-eth-resume-alice-down-bob-redeems (29526ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-funds (31178ms)
    ✓ rfc003-btc-eth-resume-bob-down-alice-redeems (30845ms)
    ✓ rfc003-btc-eth-alice-underfunds-bob-aborts (28560ms)
    ✓ rfc003-btc-eth-bob-underfunds-both-refund (30381ms)
    ✓ rfc003-btc-eth-alice-overfunds-bob-aborts (28593ms)
    ✓ rfc003-btc-eth-bob-overfunds-both-refund (30650ms)
  E2E: Bitcoin/bitcoin - Ethereum/erc20
    ✓ rfc003-btc-eth-erc20-alice-redeems-bob-redeems (31354ms)
    ✓ rfc003-btc-eth-erc20-bob-refunds-alice-refunds (29265ms)

Test Suites: 1 failed, 7 passed, 8 total
Tests:       42 passed, 42 total
Snapshots:   0 total
Time:        423.744s
Ran all test suites.
Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 27080
A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.
Found pid file bitcoind/miner.pid, sending SIGINT to process with PID 27191
Found pid file parity/parity.pid, sending SIGINT to process with PID 27075
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?
error Command failed with exit code 1.

[thomaseizinger]

I ran them twice with --maxWorkers=2 now and it always finished successfully :)

[da-kami]

Ran it once, all green.
Ran it again and the lnd_sanity test failed.

Seems the first, successful, run on the latest commit of the branch did not properly clean-up lnd, partiy, bitcoind.

Cleaned up, ran again, and the lnd_sanity test failed again at E2E: Sanity - LND Alice pays Bob › sanity-lnd-alice-pays-bob-using-hold-invoice.
I noticed that lnd_sanity ran after a bunch of other tests had already passed.

Do we share the lnd instance in all the tests? Could that cause trouble with the lnd instance not being in the right state for he sanity test when it runs last?

Note: After the last, unsuccessful run bitcoind, partiy, lnd were not properly cleaned up.

[thomaseizinger]

Note: After the last, unsuccessful run bitcoind, partiy, lnd were not properly cleaned up.

That is expected.

The lnd tests will fail if the instances are not cleanup properly because the new chain that lnd is connected to will be different every test.

[thomaseizinger]

@da-kami can you run them repeatedly if you clean the instances manually between the test runs?

[da-kami]

@da-kami can you run them repeatedly if you clean the instances manually between the test runs?

I did clean them up after the second (unsuccessful) run, but then the third run still failed.

[tcharding]

I ran yarn test maxworkers=2 3 times, 1 time passed other 2 failed. I ran yarn test maxworkers=5 once, it failed. Output was:

yarn test --maxworkers=2
yarn run v1.22.4
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
$ jest --forceExit --maxworkers=2
 PASS  tests/rfc003_schema.ts (19s)
 PASS  tests/sanity.ts (22.904s)
 PASS  tests/peers_using_ip.ts (7.423s)
 PASS  tests/multiple_peers.ts
 PASS  tests/ethereum_bitcoin.ts (100.865s)
 PASS  tests/bitcoin_ethereum_restart_cnds.ts (111.64s)
 PASS  tests/lightning_routes.ts (6.777s)
 PASS  tests/lnd_sanity.ts (17.313s)
 PASS  tests/erc20_bitcoin.ts (65.283s)
 PASS  tests/bitcoin_ethereum.ts (214.779s)
 PASS  tests/bitcoin_erc20.ts (65.972s)

Test Suites: 11 passed, 11 total
Tests:       43 passed, 43 total
Snapshots:   0 total
Time:        219.671s
Ran all test suites.
Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 11699
A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.
Found pid file bitcoind/miner.pid, sending SIGINT to process with PID 11803
Found pid file lnd-alice/lnd.pid, sending SIGINT to process with PID 12531
Found pid file lnd-bob/lnd.pid, sending SIGINT to process with PID 12533
Found pid file parity/parity.pid, sending SIGINT to process with PID 11698
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?
Done in 226.79s.
tobin@ares|~/build/github.com/comit-network/comit-rs/api_tests on 2191-parallel-e2e-tests
± htop
tobin@ares|~/build/github.com/comit-network/comit-rs/api_tests on 2191-parallel-e2e-tests
± yarn test --maxworkers=5
yarn run v1.22.4
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 0.24s
$ jest --forceExit --maxworkers=5
 PASS  tests/ethereum_bitcoin.ts (96.873s)
 PASS  tests/bitcoin_ethereum_restart_cnds.ts (133.162s)
 PASS  tests/bitcoin_erc20.ts (71.577s)
 PASS  tests/sanity.ts (14.668s)
 PASS  tests/erc20_bitcoin.ts (60.483s)
 PASS  tests/rfc003_schema.ts (11.532s)
 PASS  tests/peers_using_ip.ts (8.39s)
 FAIL  tests/bitcoin_ethereum.ts (205.764s)
  ● E2E: Bitcoin/bitcoin - Ethereum/ether › rfc003-btc-eth-bob-underfunds-both-refund

    Transaction nonce is too low. Try incrementing the nonce.

      at getResult (node_modules/ethers/providers/json-rpc-provider.js:40:21)
      at exports.XMLHttpRequest.request.onreadystatechange (node_modules/ethers/utils/web.js:111:30)
      at exports.XMLHttpRequest.dispatchEvent (node_modules/xmlhttprequest/lib/XMLHttpRequest.js:591:25)
      at setState (node_modules/xmlhttprequest/lib/XMLHttpRequest.js:610:14)
      at IncomingMessage.<anonymous> (node_modules/xmlhttprequest/lib/XMLHttpRequest.js:447:13)

Test Suites: 1 failed, 7 passed, 8 of 11 total
Tests:       1 failed, 35 passed, 36 total
Snapshots:   0 total
Time:        207.779s, estimated 215s
Ran all test suites.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
-> [1]
tobin@ares|~/build/github.com/comit-network/comit-rs/api_tests on 2191-parallel-e2e-tests
± yarn test --maxworkers=2
yarn run v1.22.4
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 0.25s
$ jest --forceExit --maxworkers=2
Determining test suites to run...Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 17898
Found pid file bitcoind/miner.pid, sending SIGINT to process with PID 17993
Found pid file lnd-alice/lnd.pid, sending SIGINT to process with PID 18792
Found pid file lnd-bob/lnd.pid, sending SIGINT to process with PID 18799
Found pid file parity/parity.pid, sending SIGINT to process with PID 17897
 PASS  tests/ethereum_bitcoin.ts (115.448s)
 PASS  tests/bitcoin_ethereum_restart_cnds.ts (120.699s)
 PASS  tests/bitcoin_erc20.ts (65.849s)
 PASS  tests/erc20_bitcoin.ts (70.906s)
 PASS  tests/sanity.ts (14.666s)
 PASS  tests/rfc003_schema.ts (12.443s)
 PASS  tests/peers_using_ip.ts (8.395s)
 PASS  tests/lnd_sanity.ts (20.447s)
 PASS  tests/multiple_peers.ts
 PASS  tests/lightning_routes.ts (6.857s)
 FAIL  tests/bitcoin_ethereum.ts (278.582s)
  ● E2E: Bitcoin/bitcoin - Ethereum/ether › rfc003-btc-eth-alice-refunds-bob-refunds

    : Timeout - Async callback was not invoked within the 123000ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 123000ms timeout specified by jest.setTimeout.Error:

      42 |     );
      43 | 
    > 44 |     it(
         |     ^
      45 |         "rfc003-btc-eth-alice-refunds-bob-refunds",
      46 |         twoActorTest(async ({ alice, bob }) => {
      47 |             await alice.sendRequest(AssetKind.Bitcoin, AssetKind.Ether);

      at new Spec (node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
      at Suite.<anonymous> (tests/bitcoin_ethereum.ts:44:5)

  ● E2E: Bitcoin/bitcoin - Ethereum/ether › rfc003-btc-eth-alice-underfunds-bob-aborts

    TimeoutError: Promise timed out after 120000 milliseconds

      at Timeout._onTimeout (node_modules/p-timeout/index.js:34:63)

Test Suites: 1 failed, 10 passed, 11 total
Tests:       2 failed, 41 passed, 43 total
Snapshots:   0 total
Time:        280.495s
Ran all test suites.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
-> [1]
tobin@ares|~/build/github.com/comit-network/comit-rs/api_tests on 2191-parallel-e2e-tests
± yarn test --maxworkers=2
yarn run v1.22.4
$ cargo build --bin cnd && tsc
    Finished dev [unoptimized + debuginfo] target(s) in 0.27s
$ jest --forceExit --maxworkers=2
Determining test suites to run...Found pid file bitcoind/bitcoind.pid, sending SIGINT to process with PID 19170
Found pid file bitcoind/miner.pid, sending SIGINT to process with PID 19265
Found pid file lnd-alice/lnd.pid, sending SIGINT to process with PID 19973
Found pid file lnd-bob/lnd.pid, sending SIGINT to process with PID 19978
Found pid file parity/parity.pid, sending SIGINT to process with PID 19169
 PASS  tests/ethereum_bitcoin.ts (103.728s)
 PASS  tests/bitcoin_ethereum_restart_cnds.ts (121.733s)
 PASS  tests/bitcoin_erc20.ts (60.728s)
 PASS  tests/sanity.ts (15.181s)
 PASS  tests/erc20_bitcoin.ts (65.837s)
 PASS  tests/rfc003_schema.ts (10.177s)
 PASS  tests/peers_using_ip.ts (9.083s)
 PASS  tests/lightning_routes.ts (7.557s)
 PASS  tests/multiple_peers.ts
 PASS  tests/lnd_sanity.ts (22.41s)
 FAIL  tests/bitcoin_ethereum.ts (278.795s)
  ● E2E: Bitcoin/bitcoin - Ethereum/ether › rfc003-btc-eth-bob-refunds-alice-refunds

    : Timeout - Async callback was not invoked within the 123000ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 123000ms timeout specified by jest.setTimeout.Error:

      25 |     );
      26 | 
    > 27 |     it(
         |     ^
      28 |         "rfc003-btc-eth-bob-refunds-alice-refunds",
      29 |         twoActorTest(async ({ alice, bob }) => {
      30 |             await alice.sendRequest(AssetKind.Bitcoin, AssetKind.Ether);

      at new Spec (node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
      at Suite.<anonymous> (tests/bitcoin_ethereum.ts:27:5)

  ● E2E: Bitcoin/bitcoin - Ethereum/ether › rfc003-btc-eth-alice-refunds-bob-refunds

    TimeoutError: Promise timed out after 120000 milliseconds

      at Timeout._onTimeout (node_modules/p-timeout/index.js:34:63)

Test Suites: 1 failed, 10 passed, 11 total
Tests:       2 failed, 41 passed, 43 total
Snapshots:   0 total
Time:        280.912s
Ran all test suites.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
-> [1]
tobin@ares|~/build/github.com/comit-network/comit-rs/api_tests on 2191-parallel-e2e-tests
± co announce-improvements
Switched to branch 'announce-improvements'
Your branch is up-to-date with 'origin/announce-improvements'.
tobin@ares|~/build/github.com/comit-network/comit-rs/api_tests on announce-improvements?
± make clippy
make: *** No rule to make target 'clippy'. Stop.
-> [2]
tobin@ares|~/build/github.com/comit-network/comit-rs/api_tests on announce-improvements?
± cd ..
tobin@ares|~/build/github.com/comit-network/comit-rs on announce-improvements?
± make clippy
rustup run --install 1.42.0 cargo --color always clippy --all-targets -- -D warnings
   Compiling cnd v0.7.2 (/home/tobin/SPOT/build/github.com/comit-network/comit-rs/cnd)
   Compiling digest-macro-derive v0.1.0 (/home/tobin/SPOT/build/github.com/comit-network/comit-rs/digest-macro-derive)
    Checking libp2p-comit v0.1.0 (/home/tobin/SPOT/build/github.com/comit-network/comit-rs/libp2p-comit)
    Checking digest v0.1.0 (/home/tobin/SPOT/build/github.com/comit-network/comit-rs/digest)
^R  Building [====================================================>  ] 541/552: cnd, cnd, swap_digest(test)                          
^CMakefile:77: recipe for target 'clippy' failed===================> ] 542/552: cnd, cnd                                             
make: *** [clippy] Interrupt
-> [130]
tobin@ares|~/build/github.com/comit-network/comit-rs on announce-improvements!?
± 

[thomaseizinger]

To me it feels like they always run successful on the first try which hints that we don't clean up properly somewhere :/

But is running them one successfully good enough for us?

[bonomat]

@thomaseizinger , @da-kami , @luckysori , @tcharding , @rishflab , @D4nte : I rebased and remove the no-mergify label. You have time to intervene until the build went through 😬

[mergify]

bors r+

[bors]

Build failed

• ci/circleci: debug-build-test

[bonomat]

bors retry

[bors]

Build succeeded

• ci/circleci: debug-build-test

[da-kami]

Hahaha, now we HAVE to deal with it :D