-
Notifications
You must be signed in to change notification settings - Fork 26.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cover 1st-party examples with tests #45270
Merged
timneutkens
merged 49 commits into
vercel:canary
from
jankaifer:jankaifer/next-333-cover-examples-of-internal-nextjs
Feb 27, 2023
Merged
Changes from all commits
Commits
Show all changes
49 commits
Select commit
Hold shift + click to select a range
df87d54
Test selected examples
jankaifer 6487a71
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer 4adcd49
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer d6b5a55
add the rest of the examples
jankaifer d539b41
TYPOs
jankaifer a1be69e
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer 07ba14d
add better logs
jankaifer daadd6b
fix using-router example
jankaifer 9c97eff
fix dep versions in example
jankaifer 0bfa744
fix outdated deps in example
jankaifer 9214bca
fix unhandled uninitialized var in example
jankaifer 9ee2286
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer 4470e8d
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer c3a4dd7
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer c1e6c23
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
kodiakhq[bot] 6e02849
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer 0a76e55
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer 2850d4f
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer c424ed6
split test into multiple files for performance reasons
jankaifer 7c92114
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer db1f050
reduce number of examples run
jankaifer 3c883e8
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer 6b33638
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer 404c026
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer 10b7f10
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
timneutkens c9a9696
wip make standalowne workflow for examples
jankaifer af2e484
refactor example tests
jankaifer e052d7f
don't execute with cron on forks
jankaifer f220ab8
better comment
jankaifer 98f5a57
wip
jankaifer 40ef8b7
removed strict equal
jankaifer c2f6966
fixed copy-pasta
jankaifer f07baa0
wip
jankaifer 7dda882
wip
jankaifer 70dd3c7
remove unused variables
jankaifer a6330b6
wip
jankaifer 0cadaef
checkout code
jankaifer 2c0e5a6
wip
jankaifer dbb865c
mov install to actin itself
jankaifer 8a869c0
fix setup node step
jankaifer 92b045f
added build step
jankaifer 8d39fbc
Enable test for all examples
jankaifer 494d995
increase timeout
jankaifer 9a17e92
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer b35c256
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer eed8fb8
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer bad9f53
refactor to use describe.each
jankaifer e2f938d
remove needles function
jankaifer 006cb1c
Merge branch 'canary' into jankaifer/next-333-cover-examples-of-inter…
jankaifer File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
# This file duplicates bunch of things from build_test_deploy | ||
|
||
on: | ||
workflow_dispatch: | ||
inputs: | ||
is_dispatched: | ||
description: 'Leave this option enabled' | ||
required: true | ||
default: true | ||
type: boolean | ||
schedule: | ||
- cron: '0 */4 * * *' | ||
|
||
name: Test examples | ||
|
||
env: | ||
PNPM_VERSION: 7.24.3 | ||
|
||
jobs: | ||
testExamples: | ||
# Don't execute using cron on forks | ||
if: (github.repository == 'vercel/next.js') || (inputs.is_dispatched == true) | ||
name: Test Examples | ||
runs-on: ubuntu-latest | ||
timeout-minutes: 120 | ||
env: | ||
NEXT_TELEMETRY_DISABLED: 1 | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
node: [16, 18] | ||
steps: | ||
- uses: actions/checkout@v3 | ||
with: | ||
fetch-depth: 25 | ||
# https://github.com/actions/virtual-environments/issues/1187 | ||
- name: tune linux network | ||
run: sudo ethtool -K eth0 tx off rx off | ||
|
||
- name: Setup node | ||
uses: actions/setup-node@v3 | ||
with: | ||
node-version: 16 | ||
check-latest: true | ||
|
||
- run: npm i -g pnpm@${PNPM_VERSION} | ||
|
||
- run: pnpm install | ||
- run: pnpm build | ||
|
||
- run: docker run --rm -v $(pwd):/work mcr.microsoft.com/playwright:v1.28.1-focal /bin/bash -c "cd /work && curl -s https://install-node.vercel.app/v${{ matrix.node }} | FORCE=1 bash && node -v && npm i -g pnpm@${PNPM_VERSION} > /dev/null && NEXT_TEST_JOB=1 NEXT_TEST_MODE=start xvfb-run node run-tests.js --type examples >> /proc/1/fd/1" | ||
name: Run test/examples |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
import { createNextDescribe } from 'e2e-utils' | ||
import path from 'path' | ||
import fs from 'fs-extra' | ||
|
||
const testedExamples = [ | ||
// Internal features | ||
'active-class-name', | ||
'amp', | ||
'amp-first', | ||
'amp-story', | ||
'api-routes', | ||
'api-routes-cors', | ||
'api-routes-middleware', | ||
'api-routes-rate-limit', | ||
'api-routes-rest', | ||
'app-dir-i18n-routing', | ||
'app-dir-mdx', | ||
'basic-css', | ||
'basic-export', | ||
'blog', | ||
'blog-starter', | ||
'catch-all-routes', | ||
'custom-routes-proxying', | ||
'custom-server', | ||
'data-fetch', | ||
'dynamic-routing', | ||
'environment-variables', | ||
'fast-refresh-demo', | ||
'head-elements', | ||
'headers', | ||
'hello-world', | ||
'hello-world-esm', | ||
'i18n-routing', | ||
'image-component', | ||
'image-legacy-component', | ||
'layout-component', | ||
'middleware', | ||
'middleware-matcher', | ||
'modularize-imports', | ||
'nested-components', | ||
'next-css', | ||
'next-forms', | ||
'progressive-render', | ||
'redirects', | ||
'remove-console', | ||
'reproduction-template', | ||
'rewrites', | ||
'script-component', | ||
'ssr-caching', | ||
'styled-jsx-with-csp', | ||
'svg-components', | ||
'using-router', | ||
'with-absolute-imports', | ||
'with-app-layout', | ||
'with-context-api', | ||
'with-env-from-next-config-js', | ||
'with-loading', | ||
'with-shallow-routing', | ||
'with-sitemap', | ||
'with-typescript', | ||
'with-typescript-types', | ||
'with-web-worker', | ||
'with-webassembly', | ||
|
||
// Library integrations that we can't break | ||
'with-jest', | ||
'with-jest-babel', | ||
'with-mdx', | ||
'with-mdx-remote', | ||
'with-tailwindcss', | ||
'with-turbopack', | ||
'with-vercel-fetch', | ||
] | ||
|
||
describe.each(testedExamples)(`example '%s'`, (example) => { | ||
// If there is an issue during a build, jest won't tell us which example caused it | ||
// we need to log it ourselfs | ||
beforeAll(() => { | ||
require('console').log(`Running example '${example}'`) | ||
}) | ||
|
||
const exampleFiles = path.join(__dirname, '..', '..', 'examples', example) | ||
const packageJson = fs.readJsonSync(path.join(exampleFiles, 'package.json')) | ||
createNextDescribe( | ||
`example '${example}'`, | ||
{ | ||
files: exampleFiles, | ||
dependencies: { | ||
// We need to make sure that these default dependencies are not installed by default | ||
// for our examples to ensure that they have all their dependencies in package.json | ||
'@types/node': undefined, | ||
'@types/react': undefined, | ||
next: undefined, | ||
react: undefined, | ||
'react-dom': undefined, | ||
typescript: undefined, | ||
...packageJson.dependencies, | ||
...packageJson.devDependencies, | ||
}, | ||
}, | ||
() => { | ||
it('builds', () => {}) | ||
} | ||
) | ||
}) |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of using an allowlist, we might want to change this to an ignorelist so that new examples are automatically tested.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since we are not running in normal CI it's actually possible now 👍
We probably should run all examples and we will see in the CI exactly which ones are broken.
But I still think that we want to have a workflow like this with only 1st party examples that should be always green.
And if we start running CI on all examples we need to be extra cautious when adding new libraries because we will be installing that library with access to our CI secrets.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will add this in Follow-up PR