Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: improve WPT runner to summarize status and update WPT #33297

Closed
wants to merge 2 commits into from

Commits on May 12, 2020

  1. test: update WPT interfaces and hr-time

    This commit updates the interfaces to
    https://github.com/web-platform-tests/wpt/tree/8ada332aea/interfaces
    and updates the hr-time test status:
    
    - `window-worker-timeOrigin.window.js` should be skipped because we
      don't implement `Blob`
    - `idlharness.any.js` should be skipped since the IDL parser needs
      to be updated, but the parser update would also result in
      an update of the test harness which in turn requires updates of
      other tests. We need to fix the URL implementation first,
      and then update the harness and all the tests.
    joyeecheung committed May 12, 2020
    Copy the full SHA
    7737b7f View commit details
    Browse the repository at this point in the history
  2. test: refactor WPTRunner

    - Print test results as soon as they are available, instead of
      until after all the tests are complete. This helps us printing
      tests whose completion callback is not called because of
      failures.
    - Run the scripts specified by `// META: script=` one by one
      instead of concatenating them first for better error stack
      traces.
    - Print a status summary when the test process is about to exit.
      This can be used as reference for updating the status file.
    
    For example the stderr output of
    `out/Release/node test/wpt/test-console.js` would be:
    
    ```
    {
      'idlharness.any.js': {
        fail: {
          expected: [
            'assert_equals: operation has wrong .length expected 1 but got 0'
          ]
        }
      }
    }
    Ran 4/4 tests, 0 skipped, 3 passed, 1 expected failures, 0 unexpected failures
    ```
    joyeecheung committed May 12, 2020
    Copy the full SHA
    366a1fc View commit details
    Browse the repository at this point in the history