Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E VSCode tests are skipped/disabled #342

Open
bd82 opened this issue Dec 17, 2020 · 4 comments
Open

E2E VSCode tests are skipped/disabled #342

bd82 opened this issue Dec 17, 2020 · 4 comments

Comments

@bd82
Copy link
Member

bd82 commented Dec 17, 2020

There seems to be multiple issues involved.

  1. These tests are consistently failing.
    • Checked on both windows/mac local dev machines.
  2. On CircleCI starting VSCode during seems to fail on the 2nd+ suite.

This root cause does not seem to be related to this repo's source code
As there are no failing builds merged to master, nor does checking-out the previous release tag resolve the issue.

There are two factors that may be related.

  • The tests run on the latest version of VSCode, so that may have changed.
  • The tests run on CircleCI docker images which are also updated automatically.

I am currently disabling some of these tests as I am unable to figure out what is the issue.

@bd82 bd82 added the Tech Debt label Dec 17, 2020
bd82 added a commit that referenced this issue Dec 17, 2020
@bd82
Copy link
Member Author

bd82 commented Dec 17, 2020

The second issue where CircleCI only succeeds in starting VSCode once may not need to be resolved
if we/when we would have a better framework for VSCode testing and will only call runTests once.

@bd82 bd82 changed the title E2E VSCode Tests for stableid are skipped E2E VSCode tests are skipped Dec 17, 2020
bd82 added a commit that referenced this issue Dec 17, 2020
bd82 added a commit that referenced this issue Dec 17, 2020
bd82 added a commit that referenced this issue Dec 17, 2020
@bd82 bd82 changed the title E2E VSCode tests are skipped E2E VSCode tests are skipped/disabled Dec 17, 2020
@bd82
Copy link
Member Author

bd82 commented Dec 17, 2020

Now encountering random inconsistent failures with the VSCode tests.
I am leaning towards disabling all these E2E tests, it is not the first time we had stability issues with these.

bd82 added a commit that referenced this issue Dec 17, 2020
bd82 added a commit that referenced this issue Dec 17, 2020
@petermuessig
Copy link
Contributor

Still valid @bd82 ?

@bd82
Copy link
Member Author

bd82 commented Aug 21, 2022

I was not impressed with the TCO of these tests.
Perhaps newer versions of the VSCode E2E testing framework should be evaluated...

@uxkjaer uxkjaer linked a pull request Oct 18, 2022 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants