Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: E2E tests against ipfs-webui HEAD #6825

Merged
merged 2 commits into from Jan 17, 2020
Merged

Conversation

lidel
Copy link
Member

@lidel lidel commented Jan 15, 2020

Parent: ipfs/ipfs-webui#1164

This PR adds CircleCI config for running ipfs-webui's end to end tests against ipfs binary produced by this repo.

It does not increase CI time, as its runs in parallel to interop tests, and those take much longer:

times

The goal is to catch regressions in go-ipfs+ipfs-webui interop, such as this one (hardened go-ipfs was unable to connect to peer that used weak key size):

Screenshot_2020-01-15 ipfs-webui (23414) - ipfs go-ipfs

When everything works as expected, it looks like this:

green

TODO

  • use custom executor capable of running Chrome in headless mode
  • fix ipfsd-ctl error with latest go-ipfs master
    • error: [test:e2e ] SyntaxError: Unexpected end of JSON input
    • reason: ipfs init failed because ipfsd-ctl uses 1024 bit keys in test mode
    • fix: set LIBP2P_ALLOW_WEAK_RSA_KEYS: true in test env
  • confirm version from IPFS_GO_EXEC is used
    • confirmed: [test:e2e ] E2E using /go-ipfs/0.5.0-dev/28a9a6501 (/tmp/circleci-workspace/bin/ipfs) with Peer ID QmXcHQoM2SFTAaxE2khR4WpPDCnouYNMPPEkqxZxKNk6oa
  • confirm CI works and added time is acceptable
    • we should be good: it takes ~6m and finishes long before interop tests suite (both run in parallel)
  • consider: if test time is low enough, run original tests first to confirm ipfs-webui's HEAD is green, then run again with ipfs from this repo's HEAD
    • decision: running tests second time take less than a minute, sounds feasible
    • note: if upstream tests fail, we dont break the build, but exit with code 0 (skipping, as there is no way to test for regression if upstream webui is broken)
  • fix failed to negotiate security protocol: MAC verification failed)
    • note: it is super cool that the moment we've set up E2E regression test, we've found a regression 👌
    • reason: LIBP2P_ALLOW_WEAK_RSA_KEYS=true did not decrease minimal key size limit everywhere
    • fix: changing to LIBP2P_ALLOW_WEAK_RSA_KEYS=1 fixed the "MAC verification failed" error. I updated true1 everywhere.
  • switch to ipfs-webui's master

@lidel lidel force-pushed the test/webui-e2e-interop branch 7 times, most recently from 6e3baac to a820e15 Compare January 16, 2020 11:49
This adds interop/regression tests against ipfs-webui repo.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
@lidel lidel marked this pull request as ready for review January 17, 2020 15:52
@lidel
Copy link
Member Author

lidel commented Jan 17, 2020

@Stebalien this is ready for review

.circleci/config.yml Outdated Show resolved Hide resolved
Co-Authored-By: Hugo Dias <hugomrdias@gmail.com>
@Stebalien Stebalien merged commit 39bf95b into master Jan 17, 2020
@Stebalien Stebalien deleted the test/webui-e2e-interop branch January 17, 2020 18:29
@Stebalien
Copy link
Member

🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants