Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document and Resolve all Flaky Tests #1771

Open
3 of 9 tasks
fuxingloh opened this issue Sep 22, 2022 · 0 comments
Open
3 of 9 tasks

Document and Resolve all Flaky Tests #1771

fuxingloh opened this issue Sep 22, 2022 · 0 comments
Labels
area/apps area/packages kind/feature New feature request triage/accepted Triage has been accepted

Comments

@fuxingloh
Copy link
Member

fuxingloh commented Sep 22, 2022

What would you like to be added:

As per the title, documenting and resolving all flaky test cases.

/area packages apps
/triage accepted

Flaky Tests:

  • FAIL apps/playground-api/tests/modules/PlaygroundModule.test.ts (113.96 s)
FAIL apps/playground-api/__tests__/modules/PlaygroundModule.test.ts (113.96 s)
  ● should have gov set

    expect(received).rejects.toThrow(expected)

    Expected substring: "Payback of loan via DUSD token is not currently active"
    Received message:   "RpcApiError: 'Test PaybackLoanTx execution failed:
    Cannot payback loan while any of the asset's price is invalid', code: -32600, method: paybackloan"

          89 |
          90 |     if (error != null) {
        > 91 |       throw new RpcApiError({
             |             ^
          92 |         ...error,
          93 |         method: method
          94 |       })

      at Function.parse (packages/jellyfish-api-jsonrpc/src/index.ts:91:13)
      at TestingJsonRpcClient.call (packages/jellyfish-api-jsonrpc/src/index.ts:69:30)
                at runMicrotasks (<anonymous>)
      at Loan.paybackLoan (packages/jellyfish-api-core/src/category/loan.ts:267:12)
      at Object.<anonymous> (apps/playground-api/__tests__/modules/PlaygroundModule.test.ts:297:3)
      at Object.toThrow (node_modules/expect/build/index.js:285:22)
      at Object.<anonymous> (apps/playground-api/__tests__/modules/PlaygroundModule.test.ts:297:33)
          at runMicrotasks (<anonymous>)
  • FAIL packages/jellyfish-transaction-builder/tests/txn/txn_builder_loan_payback_loan.test.ts (366.098 s)
FAIL packages/jellyfish-transaction-builder/__tests__/txn/txn_builder_loan_payback_loan.test.ts (366.098 s)
  ● paybackLoan success › should paybackLoan by anyone

    expect(received).toStrictEqual(expected) // deep equality

    Expected: "0.00006850"
    Received: "0.00004567"

      448 |     const tslaInterestTotal = tslaInterestsPerBlockBefore.multipliedBy(height - tslaLoanHeight + 1)
      449 |     expect(interestsBeforePayback[0].interestPerBlock.toFixed(8)).toStrictEqual(tslaInterestsPerBlockBefore.toFixed(8, BigNumber.ROUND_CEIL))
    > 450 |     expect(interestsBeforePayback[0].totalInterest.toFixed(8)).toStrictEqual(tslaInterestTotal.toFixed(8, BigNumber.ROUND_CEIL))
          |                                                                ^
      451 |
      452 |     const tslaLoanAmountBefore = new BigNumber(40).plus(tslaInterestTotal).decimalPlaces(8, BigNumber.ROUND_CEIL)
      453 |     const tslaLoanValueBefore = tslaLoanAmountBefore.multipliedBy(2)

      at Object.<anonymous> (packages/jellyfish-transaction-builder/__tests__/txn/txn_builder_loan_payback_loan.test.ts:450:64)
          at runMicrotasks (<anonymous>)
  • FAIL packages/jellyfish-api-core/tests/category/account/getPendingFutureSwaps.test.ts (117.119 s)
FAIL packages/jellyfish-api-core/__tests__/category/account/getPendingFutureSwaps.test.ts (117.119 s)
  ● Account GetPendingFutureSwaps › Single futureswap › If GOV attributes: token is disabled after futureswap › Should getPendingFutureSwaps if futureswap DUSD for TSLA

    RpcApiError: 'Test SetGovVariableTx execution failed:
    tx not from foundation member', code: -32600, method: setgov

      89 |
      90 |     if (error != null) {
    > 91 |       throw new RpcApiError({
         |             ^
      92 |         ...error,
      93 |         method: method
      94 |       })

      at Function.parse (packages/jellyfish-api-jsonrpc/src/index.ts:91:13)
      at TestingJsonRpcClient.call (packages/jellyfish-api-jsonrpc/src/index.ts:69:30)
          at runMicrotasks (<anonymous>)
      at Masternode.setGov (packages/jellyfish-api-core/src/category/masternode.ts:177:12)
      at setup (packages/jellyfish-api-core/__tests__/category/account/getPendingFutureSwaps.test.ts:126:5)
      at Object.<anonymous> (packages/jellyfish-api-core/__tests__/category/account/getPendingFutureSwaps.test.ts:144:5)
  • FAIL packages/jellyfish-api-core/tests/category/loan/placeAuctionBid.test.ts (247.409 s)
FAIL packages/jellyfish-api-core/__tests__/category/loan/placeAuctionBid.test.ts (247.409 s)
  ● placeAuctionBid success › should placeAuctionBid with utxos

    RpcApiError: 'Insufficient funds', code: -4, method: placeauctionbid
  • FAIL packages/jellyfish-transaction-builder/tests/txn/txn_builder_account_future_swap.test.ts (341.543 s)
FAIL packages/jellyfish-transaction-builder/__tests__/txn/txn_builder_account_future_swap.test.ts (341.543 s)
  ● withdraw futureswap › should withdraw futureswap dusd to dtoken

    RpcApiError: 'Test SetOracleDataTx execution failed:
    Timestamp (1663829748) is out of price update window (median: 1579045229)', code: -32600, method: setoracledata
  • FAIL packages/jellyfish-api-core/tests/category/loan/withdrawFromVault.test.ts (194.061 s)
FAIL packages/jellyfish-api-core/__tests__/category/loan/withdrawFromVault.test.ts (194.061 s)
  ● withdrawFromVault with 50% DUSD or DFI collaterals › should not takeLoan with 33.33% DUSD collateral

    expect(received).rejects.toThrow(expected)

    Expected substring: "At least 50% of the minimum required collateral must be in DFI or DUSD"
    Received message:   "RpcApiError: 'WithdrawFromVaultTx: Vault does not have enough collateralization ratio defined by loan scheme - 133 < 200 (code 16)', code: -26, method: withdrawfromvault"
  • FAIL packages/jellyfish-wallet-encrypted/tests/scrypt.test.ts (8.208 s)
FAIL packages/jellyfish-wallet-encrypted/__tests__/scrypt.test.ts (8.208 s)
  ● configurable params (easy-hard)

    expect(received).toBeGreaterThan(expected)

    Expected: > 2990
    Received:   2667

      33 |   // significantly slower
      34 |   // technically it is 8x harder, but they can be processed in parallel
    > 35 |   expect(hardTime).toBeGreaterThan(easyTime * 2)
         |                    ^
      36 | })
      37 |

      at Object.<anonymous> (packages/jellyfish-wallet-encrypted/__tests__/scrypt.test.ts:35:20)
  • FAIL apps/playground-api/tests/controllers/WalletController.test.ts (120.766 s)
FAIL apps/playground-api/__tests__/controllers/WalletController.test.ts (120.766 s)
  ● sendUtxo › should send utxo to address and wait for automated block confirmation

    expect(received).toStrictEqual(expected) // deep equality

    Expected: 1
    Received: 0

      31 |     })
      32 |
    > 33 |     expect(unspent.length).toStrictEqual(1)
         |                            ^
      34 |     expect(unspent[0].address).toStrictEqual(address)
      35 |     expect(unspent[0].amount).toStrictEqual(new BigNumber('19.34153143'))
      36 |   })

      at Object.<anonymous> (apps/playground-api/__tests__/controllers/WalletController.test.ts:33:28)
          at runMicrotasks (<anonymous>)
  • FAIL packages/jellyfish-api-core/tests/category/blockchain/getMempoolDescendants.test.ts (28.545 s)
FAIL packages/jellyfish-api-core/__tests__/category/blockchain/getMempoolDescendants.test.ts (28.545 s)
  ● Transactions with descendants › should return array of transaction ids if verbose is false

    expect(received).toBeGreaterThan(expected)

    Expected: > 0
    Received:   0

       95 |     const txIdWithDescendants = await getTxIdWithDescendants()
       96 |     const mempoolDescendants = await testing.rpc.blockchain.getMempoolDescendants(txIdWithDescendants, false)
    >  97 |     expect(mempoolDescendants.length).toBeGreaterThan(0)
          |                                       ^
       98 |     for (const descendantId of mempoolDescendants) {
       99 |       expect(descendantId).toStrictEqual(expect.stringMatching(/^[0-9a-f]{64}$/))
      100 |     }

      at Object.<anonymous> (packages/jellyfish-api-core/__tests__/category/blockchain/getMempoolDescendants.test.ts:97:39)
          at runMicrotasks (<anonymous>)
@fuxingloh fuxingloh added the kind/feature New feature request label Sep 22, 2022
@fuxingloh fuxingloh pinned this issue Jan 24, 2023
fuxingloh added a commit that referenced this issue Jan 25, 2023
…th a deterministic setup (#1997)

#### What this PR does / why we need it:

Use a deterministic setup for `getMempoolDescendants` to prevent test
flakiness.

Fixes part of #1771
fuxingloh added a commit that referenced this issue Jan 25, 2023
…rate "addressB" UTXO (#1772)

#### What this PR does / why we need it:

As per the title.

#### Which issue(s) does this PR fixes?:
<!--
(Optional) Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
-->
Fixes part of #1771
fuxingloh added a commit that referenced this issue Jan 25, 2023
… by using `waitForExpect` (#1999)

#### What this PR does / why we need it:

Add `waitForExpect` for `wallet.listUnspent` within
`WalletController.test.ts` as it's known to be "flaky" from testing;
wallet operations run differently on a separate wallet subsystem from
blockchain core `connectBlock`, a block might get connected first before
your wallet entries get indexed.

#### Which issue(s) does this PR fixes?:

Fixes part of #1771
fuxingloh added a commit that referenced this issue Jan 31, 2023
…syTime` condition to reduce test flakiness (#1998)

#### What this PR does / why we need it:

I would forgo this test completely (if this is still causing flakiness)
since we can't control (or should not) the CI environment. Tests
concurrency within the pipeline (Jest concurrent and Docker spinup) skew
"hardTime" and "easyTime" making the test number very unreliable.

We could isolate the tests but frankly too much trouble to prove an
otherwise proven theory.

#### Which issue(s) does this PR fixes?:

Fixes part of #1771
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/apps area/packages kind/feature New feature request triage/accepted Triage has been accepted
Projects
None yet
Development

No branches or pull requests

2 participants