Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

test: E2E tests against ipfs-webui HEAD #2706

Merged
merged 1 commit into from Feb 13, 2020
Merged

test: E2E tests against ipfs-webui HEAD #2706

merged 1 commit into from Feb 13, 2020

Conversation

lidel
Copy link
Member

@lidel lidel commented Jan 8, 2020

Parent: ipfs/ipfs-webui#1164
go-ipfs counterpart: ipfs/kubo#6825

This PR adds TravisCI config for running ipfs-webui's end to end tests against js-ipfs from this repo.

It does not increase CI time, as its runs in parallel to other tests, and those take much longer:

-22020-03-03--12-35-58

Note: It uses aegir test-external which runs tests twice: first time against release version of js-ipfs, and if that is green, then against version from this repo. If not, we exit without error. That way we avoid breaking the js-ipfs build if upstream ipfs-webui is not green.

TODO

  • webui needs build before e2e runx
  • ensure binary from HEAD is used during second time webui's test is run
  • there is false-positive bug that needs to be fixed
    • suspect: aegir test-external returns 0 if test with original js-ipfs fails its by design ;)

cc @autonome @hugomrdias @alanshaw

@lidel lidel force-pushed the test/webui-e2e branch 3 times, most recently from fe04356 to 09f491e Compare January 10, 2020 15:08
@lidel
Copy link
Member Author

lidel commented Jan 13, 2020

Getting late, wrote some notes below to pick this up tomorrow.

Using wrong version of js-ipfs

Something felt off. Just to be sure we test the right thing, I updated js-ipfs version in package.json to end with -test and run test-external

$ E2E_IPFSD_TYPE=js npm run test:external -- ipfs-webui https://github.com/ipfs-shipyard/ipfs-webui.git --branch "fix/test-external"

This revealed a false-positive: for some reason the second time npm test is run during aegir text-external it does not switch to HEAD version of js-ipfs.

  • Expected behavior would be to use js-ipfs injected via connect-deps to package.json (.connect-deps-cache/ipfs-0.40.0-test*.tgz)

  • Instead, the require.resolve('ipfs/src/cli/bin.js') returns /tmp/ipfs-webui-{random}/node_modules/ipfs/src/cli/bin.js which belongs to old js-ipfs (yarn pack used by connect-deps seems to update only the manifest at /tmp/ipfs-webui-{random}/node_modules/ipfs/package.json):

    [test:e2e  ] E2E using js-ipfs/0.40.0 (/tmp/ipfs-webui-{random}/node_modules/ipfs/src/cli/bin.js)
    ...
    [test:e2e  ] E2E using js-ipfs/0.40.0 (/tmp/ipfs-webui-{random}/node_modules/ipfs/src/cli/bin.js)
    

Defining explicit IPFS_JS_EXEC solves the problem:

$ E2E_IPFSD_TYPE=js IPFS_JS_EXEC=$(realpath ./src/cli/bin.js) npm run test:external -- ipfs-webui https://github.com/ipfs-shipyard/ipfs-webui.git --branch "fix/test-external"

However, it comes at a price of executing both npm test runs against binary from IPFS_JS_EXEC :

[test:e2e  ] E2E using js-ipfs/0.40.0-test (/home/lidel/project/js-ipfs/src/cli/bin.js)
...
[test:e2e  ] E2E using js-ipfs/0.40.0-test (/home/lidel/project/js-ipfs/src/cli/bin.js)

False-positive on error during first npm test of test-external:

I've added throw new Error('TODO this is test of e2e error') to src/core/components/id.js to ensure tests fail during ipfs.id, however if first npm test run by test-external fails, the process ends with exit code 0 (success).

This seems to be a feature: we don't want to break js-ipfs build if upstream tests of 3rd party repo are broken already, we want to fail build only if upstream is green but switching to HEAD triggers error.

In short, this should be solved if we find a way to fix the first issue.

@lidel
Copy link
Member Author

lidel commented Jan 17, 2020

ipfs-webui e2e is green now: https://travis-ci.com/ipfs/js-ipfs/jobs/276855690
(thanks to @hugomrdias for the fix in ipfsd-ctl v1.0.6)

@alanshaw this is ready for review

@lidel lidel requested a review from alanshaw January 17, 2020 15:50
@lidel lidel marked this pull request as ready for review January 17, 2020 15:50
@alanshaw
Copy link
Member

@lidel would you mind rebasing so we can double check this branch is still passing?

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
@lidel
Copy link
Member Author

lidel commented Feb 12, 2020

@alanshaw had to apply small tweaks (ipfs/ipfs-webui@0fc43a8, ipfs/ipfs-webui@e139736) but its green again (works with both old and new API) 👌

@alanshaw alanshaw merged commit 34a3b3a into master Feb 13, 2020
@alanshaw alanshaw deleted the test/webui-e2e branch February 13, 2020 12:21
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants