Jest + Playwright Testing (#1276)

* Initial Jest+Playwright setup

* Fix eslint warnings

* Add placeholder for common options

* Fix load order of scriptURLs

* Add docsifyURL and waitForSelector options

* Add executeScript scenarios

* Enable firefox and webkit tests by default

* Prevent prettier from reformatting browsers list

* Update options and add JSDoc comments

* Initial commit

* Complete initial example tests

* Minor tweaks

* Change suite name

* Rename file

* Add npm script to run jest+playwright example test

* Clean up server properties

* Isolate e2e, integration, and unit test environments

- Setup projects for e2e, integration, and unit tests in Jest configuration
- Setup /tests/e2e/ tests for Jest+PlayWright
- Setup /tests/integration/ tests for Jest
- Setup /tests/unit/ tests for Jest
- Setup eslint for Jest and Jest+Playwright environments
- Move e2e configuration files into separate folder
- Update e2e example tests
- Update unit example tests
- Update HTML fixtures

* Update docsifyInit helper

- Add `debug` option
- Append `Markdown` suffix to markdown-related options
- Reorder options alphabetically

* Add jestPlaywright.debug() to e2e examples

* Removed need to pass page as arg to docsifyInit()

* Add VSCode debug configurations

* Change test server port from 8080 to 3001

* Clean up test names and expect() order

* Update npm scripts to ignore example tests

* Add CLI commands and resources section

* Add Jest unit test snapshot example

* Added Jest unit and e2e tests to Github workflows

* Add npm script to run Jest examples

Added for new devs who are exploring the examples, as the need for passing --testPathIgnorePatterns is not obvious. This flag is required because the example tests are excluded by default.

* Remove node 10.x and add 14.x from tests

Required for jest+playwright testitng

* Temporarily disable testPathIgnorePatterns for ci

* Temporarily remove node 14 from matrix

Cypress fails on node 14.x

* Alternate workflow setup for new test env

* Update workflow platforms and node versions

Also cleaning up naming

* Restoring original workflow names

* Remove Cypress tests

* Remove/Reset GitHub actions

* Restore/reset workflows

* Bump actions/checkout@v1 to v2

* Use playright GH action

* Update playwright GH action configuration

* Remove unnecessary steps and update checkout version

* Add install step

* Add --ignore-scripts flag

Linting does not require running scripts automatically executred after install.

* Swicth from ci to i for install

* Add new Build & Test workflow

* Add OS tests

* Wait for network idle for more reliable ready state

* Configure image snapshot testing and add example

* Remove example fixture file

* Adjust image snapshot threshold for local & CI tests

* Upload diff-output artifact

* Add failure() check to upload-artifact

* Bump image snapshot threshold for local & CI tests

* Set diffDirection

* Fix XHR caching in playwright tests

* Update image snapshot example and theshold

* Bump image snapshot threshold for local & CI tests

* Remove old workflows

* Bump image snapshot threshold for local & CI tests

* Bump image snapshot failureThreshold for local & CI tests

* Set failureThresholdType to percent

* Change image snapshot comparisonMethod to ssim

* Remove pixelMatch options (incompatible with ssim)

* Bump image snapshot failureThreshold

* Bump image snapshot failureThreshold

* Disable fast-fail to allow all checks to complete

* Rename workflow

* Disable fast-fail to allow all checks to complete

* Store pixel and ssim comparison configs

* Add link to image snapshot test

* Fix CI errors on Windows due to image size mismatch

* Attempt to fix macos e2e exit code 134 issue

* Update test.yml

* Update test.yml

* Update test.yml

* Fix --ci flag

* Set Jets to use --runInBand for macOS e2e

* Remove unnecessary macOS check

* Set image snapshot to runInProcess (macOS CI fix)

* Update test.yml

* Temporary failureThreshold change for debugging

* Upload os+node-specific diff artifacts

* Remove node version from diff artifact

* Revert "Remove node version from diff artifact"

This reverts commit 9cfcc43.

* Revert "Revert "Remove node version from diff artifact""

This reverts commit ad6c189.

* Restoring failureThreshold after debugging

* Remove runInProcess option

* Revert "Remove runInProcess option"

This reverts commit 667ed6c.

* Add node-specific artifact uploads

* Set ssim mode to fast and restore runInProcess

* Set failureThreshold to 0.01

* Updating to playwright@next for webkit fix

* Restore optimal ssim configuration

* Testing pixel vs. ssim image comparison

* Move shared test setup to unit/int/e2e setups

* Refactor test server setup

* Replace live-server with browser-sync

* Update script for running local docs site

- Only init GA plugin on public site
- Only init Gitter plugin on public site
- Only init Matomo plugin on public site

NOTE: Large diff is a result of initial Prettier formatting. Actual change limited to last <script> block on page.

* Allow launching test server preview with --start

* Add integration tests and refactor unit + e2e

* Unify docsifyInit() and cleanup

- Create unified docsifyInit() for jest and playwright
- Move shared helpers to /tests/helpers
- Update tests
- Update globals

* Fixed webkit routes by specifying ContentType

* Update snapshot

* Update dependencies

* Update tests to use unified docsifyInit()

* Remove Cypress (old e2e test environment)

* Update tests to run (unit/integration/e2e)

- Add new integration tests (Jest)
- Remove old unit tests (mocha+jsdom)
- Remove old e2e tests (Cypress)

* Remove Cypress-related dependency

* Remove mocha+chai+jsdom (old unit/int test env)

* Rename testing directories and scripts

- Rename /tests/ directory to /test/
- Rename script/task names by removing “jest” identifier
- Remove “test:jest-examples” script

* - Configure test server for availability with all tests (previously e2e only)
- Create identical docsifyInit() tests using Jest (integration) and Playwright (e2e)
- Update docsifyInit() to convert relative URLs to absolute URLs to work in both JSDOM and Playwright
- Update docsifyInit() to append style- and js-related tags using createElement instead of insertAdjacentHTML
- Update paths in test files to use unified docsifyInit()
- Added option to docsifyInit() to enable/disable formatted HTML output to console
- Removed vue.css as default docsify theme from docsifyInit()
- Removed outdated files

* Reorganize test files

* Fix basePath option

* Replace do-mock-ajax with xhr-mock

- Allows mocking all XMLHttpRequests instead of just those initiated via /src/core/fetch/ajax.js
- Allows JSDOM tests to use /lib/docsify.js instead of /src/core/index.js (same as Playwright tests)
- Allows JSDOM tests to use /docs site as test content

* Added new waitFor helpers

* Clean up globals

- Import globals from various files instead of manually adding them to ensure they are availability in Jest and eslint configurations
- Add middleware to server configuration for serving virtual “_blank.html” file
- Add BLANK_URL
- Rename TEST_URL to TEST_HOST
- Removed ./test/fixtures/ directory (blank page now served via server.js middleware)
- Added page.goto(BLANK_URL) call to global Playwright beforeEach() setup

* Add try/catch for waitForFunction callback

* Move playwright config into jest.config.js

* Add runInBand option to tests for reliability

* Remove unnecessary XHR Mock teardown

* Add —runInBand to test script

* Merge develop

* Cleanup

- Relocate carbon plugin script with other scripts
- Update zh-cn docs URL to align with other translatins (GitHub, not jsdelivr)
- Add major version locks to URLs

* Removed fixed host value from docs

* Updated test after merge (docsify version change)

* Added startPath for manual instance

* Remove Node 10 from CI tests

* Remove Node 10 from CI lint tests

.eslintignore

@@ -3,9 +3,8 @@ packages/docsify-server-renderer/build.js
 node_modules
 build
 server.js
-cypress
 lib
 themes
 build
 docs/
-**/*.md
+**/*.md

.eslintrc.js

@@ -6,7 +6,6 @@ module.exports = {
     ecmaVersion: 2019,
   },
   env: {
-    jest: true,
     browser: true,
     node: true,
     es6: true,

.github/workflows/lint.yml

@@ -1,31 +1,30 @@
-name: Linting Checks
+name: Lint
 
-on: 
+on:
   push:
-   branches:
-    - master
-    - develop
+    branches:
+      - master
+      - develop
   pull_request:
-   branches:
-    - master
-    - develop
+    branches:
+      - master
+      - develop
 
 jobs:
   build:
-    runs-on: ubuntu-16.04
+    runs-on: ubuntu-latest
     strategy:
+      fail-fast: false
       matrix:
-        node-version: [10.x, 12.x, 13.x]
+        node-version: [12.x, 14.x]
 
     steps:
-    - uses: actions/checkout@v1
-    - name: Use Node.js ${{ matrix.node-version }}
-      uses: actions/setup-node@v1
-      with:
-        node-version: ${{ matrix.node-version }}
-    - name:  bootstrap
-      run: npm run bootstrap
-    - name: Build
-      run: npm run build
-    - name: Linting
-      run: npm run lint
+      - uses: actions/checkout@v2
+      - name: Use Node.js ${{ matrix.node-version }}
+        uses: actions/setup-node@v1
+        with:
+          node-version: ${{ matrix.node-version }}
+      - name: Install
+        run: npm i --ignore-scripts
+      - name: Lint
+        run: npm run lint

.github/workflows/test.yml

@@ -0,0 +1,42 @@
+name: Build & Test
+
+on:
+  push:
+    branches:
+      - master
+      - develop
+  pull_request:
+    branches:
+      - master
+      - develop
+
+jobs:
+  build:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        node-version: [12.x, 14.x]
+        os: ['macos-latest', 'ubuntu-latest', 'windows-latest']
+
+    steps:
+      - uses: actions/checkout@v2
+      - name: Use Node.js ${{ matrix.node-version }}
+        uses: actions/setup-node@v1
+        with:
+          node-version: ${{ matrix.node-version }}
+      - name: bootstrap
+        run: npm run bootstrap
+      - name: unit tests
+        run: npm run test:unit -- -ci --runInBand
+      - name: integration tests
+        run: npm run test:integration -- -ci --runInBand
+      - uses: microsoft/playwright-github-action@v1.3.0
+      - name: e2e tests
+        run: npm run test:e2e -- --ci --runInBand
+      - name: Upload artifacts (diff output)
+        uses: actions/upload-artifact@v2
+        if: failure()
+        with:
+          name: ${{ matrix.os }}-${{ matrix.node-version }}-diff-output
+          path: ${{ github.workspace }}/test/**/__diff_output__/*

.gitignore

@@ -1,11 +1,10 @@
 *.log
 .DS_Store
 .idea
+__diff_output__
+lib/
 node_modules
-/themes/
-/lib/
-/cypress/integration/examples
-/cypress/fixtures/docs
+themes/
 
 # exceptions
 !.gitkeep

.vscode/launch.json

@@ -0,0 +1,73 @@
+{
+  // Use IntelliSense to learn about possible attributes.
+  // Hover to view descriptions of existing attributes.
+  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+  "version": "0.2.0",
+  "configurations": [
+    {
+      "type": "node",
+      "request": "launch",
+      "name": "Run tests",
+      "runtimeExecutable": "npm",
+      "runtimeArgs": ["run-script", "test"],
+      "console": "integratedTerminal"
+    },
+    {
+      "type": "node",
+      "request": "launch",
+      "name": "Run current test file",
+      "runtimeExecutable": "npm",
+      "runtimeArgs": ["run-script", "test"],
+      "args": ["--", "-i", "${relativeFile}", "--testPathIgnorePatterns"],
+      "console": "integratedTerminal"
+    },
+    {
+      "type": "node",
+      "request": "launch",
+      "name": "Run selected test name",
+      "runtimeExecutable": "npm",
+      "runtimeArgs": ["run-script", "test"],
+      "args": [
+        "--",
+        "-i",
+        "${relativeFile}",
+        "-t",
+        "${selectedText}",
+        "--testPathIgnorePatterns"
+      ],
+      "console": "integratedTerminal"
+    },
+    {
+      "type": "node",
+      "request": "launch",
+      "name": "Update current test file snapshot(s)",
+      "runtimeExecutable": "npm",
+      "runtimeArgs": ["run-script", "test"],
+      "args": [
+        "--",
+        "-i",
+        "${relativeFile}",
+        "--updateSnapshot",
+        "--testPathIgnorePatterns"
+      ],
+      "console": "integratedTerminal"
+    },
+    {
+      "type": "node",
+      "request": "launch",
+      "name": "Update selected test name snapshot(s)",
+      "runtimeExecutable": "npm",
+      "runtimeArgs": ["run-script", "test"],
+      "args": [
+        "--",
+        "-i",
+        "${relativeFile}",
+        "-t",
+        "${selectedText}",
+        "--updateSnapshot",
+        "--testPathIgnorePatterns"
+      ],
+      "console": "integratedTerminal"
+    }
+  ]
+}

babel.config.js

@@ -0,0 +1,12 @@
+module.exports = {
+  presets: [
+    [
+      '@babel/preset-env',
+      {
+        targets: {
+          node: 'current',
+        },
+      },
+    ],
+  ],
+};