Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run test262 again #15001

Merged
merged 10 commits into from Oct 6, 2022
Merged

Run test262 again #15001

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
1eb3b48
Run test262 again
nicolo-ribaudo Sep 29, 2022
9c0e6ce
More threads
nicolo-ribaudo Oct 1, 2022
6c3347c
Run in parallel jobs
nicolo-ribaudo Oct 1, 2022
c5564e4
Fix merging
nicolo-ribaudo Oct 3, 2022
23f12af
Fix
nicolo-ribaudo Oct 3, 2022
1e63a93
More parallel
nicolo-ribaudo Oct 3, 2022
3d7b737
Fix "Download previous results", don't compare yet
nicolo-ribaudo Oct 4, 2022
e7ec1d0
Update .github/workflows/ci.yml
nicolo-ribaudo Oct 4, 2022
cc7a51e
Try running macos which has 3 cores instead of 2
nicolo-ribaudo Oct 5, 2022
a1b8bc4
Revert "Try running macos which has 3 cores instead of 2"
nicolo-ribaudo Oct 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
130 changes: 130 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Expand Up @@ -484,3 +484,133 @@ jobs:
run: yarn up eslint@7.5.0
- name: Run babel/eslint tests
run: node ./node_modules/.bin/jest eslint

test262-prepare:
name: test262 - Prepare for running in parallel
runs-on: ubuntu-latest
steps:
- name: Checkout Babel
uses: actions/checkout@v3
- name: Use Node.js latest
uses: actions/setup-node@v3
- name: Checkout test runner
uses: actions/checkout@v3
with:
repository: babel/babel-test262-runner
path: babel-test262-runner
- name: Download test262
run: make bootstrap-test262
- name: Prepare test262 chunks
run: |
cd babel-test262-runner
npm ci
node lib/prepare-chunks 8 > ~/test262-chunks.json
env:
TEST262_PATH: ../build/test262
- name: Upload chunks artifact
uses: actions/upload-artifact@v3
with:
name: test262-chunks
path: ~/test262-chunks.json
retention-days: 3

test262:
name: test262
strategy:
matrix:
chunk: [0, 1, 2, 3, 4, 5, 6, 7]
runs-on: ubuntu-latest
nicolo-ribaudo marked this conversation as resolved.
Show resolved Hide resolved
needs: [build, test262-prepare]
steps:
- name: Checkout Babel
uses: actions/checkout@v3
- name: Use Node.js latest
uses: actions/setup-node@v3
with:
node-version: "*"
cache: "yarn"
- name: Install
run: yarn install
- uses: actions/download-artifact@v3
with:
name: babel-artifact
- name: Checkout test runner
uses: actions/checkout@v3
with:
repository: babel/babel-test262-runner
path: babel-test262-runner
- name: Install test runner
run: |
cd babel-test262-runner
npm ci
node lib/download-node
- name: Download test262
run: make bootstrap-test262
- name: Download chunks file
uses: actions/download-artifact@v3
with:
name: test262-chunks
- name: Run test262
run: |
cd babel-test262-runner
node lib/run-tests I_AM_SURE | tee ~/test262-${{ matrix.chunk }}.tap
env:
BABEL_PATH: ".."
TEST262_PATH: ../build/test262
THREADS: 4
CHUNKS_FILE: ../test262-chunks.json
CHUNK: ${{ matrix.chunk }}
- name: Create artifact with report results
uses: actions/upload-artifact@v3
with:
name: test262-result-${{ matrix.chunk }}
path: ~/test262-${{ matrix.chunk }}.tap
retention-days: 3

test262-finish:
name: test262 - Analyze results
runs-on: ubuntu-latest
needs: test262
steps:
- uses: actions/download-artifact@v3
with: { name: test262-result-0 }
- uses: actions/download-artifact@v3
with: { name: test262-result-1 }
- uses: actions/download-artifact@v3
with: { name: test262-result-2 }
- uses: actions/download-artifact@v3
with: { name: test262-result-3 }
- uses: actions/download-artifact@v3
with: { name: test262-result-4 }
- uses: actions/download-artifact@v3
with: { name: test262-result-5 }
- uses: actions/download-artifact@v3
with: { name: test262-result-6 }
- uses: actions/download-artifact@v3
with: { name: test262-result-7 }
- name: Merge chunk results
run: |
cat test262-0.tap test262-1.tap test262-2.tap test262-3.tap \
test262-4.tap test262-5.tap test262-6.tap test262-7.tap \
| npx tap-merge > test262.tap
- name: Create artifact with report results
uses: actions/upload-artifact@v3
with:
name: test262-result
path: test262.tap
- name: Checkout test runner
uses: actions/checkout@v3
with:
repository: babel/babel-test262-runner
path: babel-test262-runner
# - name: Download previous results
# run: |
# cd babel-test262-runner
# npm ci
# node lib/download-main-artifact/index.mjs ~/test262-main.tap
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# - name: Compare previous main branch & current results
# run: |
# cd babel-test262-runner
# node lib/compare-results ~/test262-main.tap ~/test262.tap | tee ~/diff.tap
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will uncomment this in a separate PR, since for now there is nothing on main to compare to.

2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Expand Up @@ -178,7 +178,7 @@ bootstrap-test262:
rm -rf build/test262
mkdir -p build
git clone --filter=blob:none --sparse --single-branch --shallow-since=2021-05-01 https://github.com/tc39/test262.git build/test262
cd build/test262 && git sparse-checkout set "test"
cd build/test262 && git sparse-checkout set "test" "harness"
cd build/test262 && git checkout -q $(TEST262_COMMIT)

test-test262:
Expand Down