id | title |
---|---|
ci |
Continuous Integration |
Playwright tests can be executed in CI environments. We have created sample configurations for common CI providers.
3 steps to get your tests running on CI:
-
Ensure CI agent can run browsers: Use our Docker image in Linux agents or install your dependencies using the CLI.
-
Install Playwright:
# Install NPM packages npm ci # or npm install # Install Playwright browsers and dependencies npx playwright install --with-deps
pip install playwright playwright install --with-deps
mvn exec:java -e -Dexec.mainClass=com.microsoft.playwright.CLI -Dexec.args="install --with-deps"
pwsh bin\Debug\netX\playwright.ps1 install --with-deps
-
Run your tests:
npm test
pytest
The Command line tools can be used to install all operating system dependencies on GitHub Actions.
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '14'
- name: Install dependencies
run: npm ci
- name: Install Playwright
run: npx playwright install --with-deps
- name: Run your tests
run: npm test
- name: Upload test results
if: always()
uses: actions/upload-artifact@v2
with:
name: playwright-report
path: playwright-report
steps:
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install playwright
pip install -e .
- name: Ensure browsers are installed
run: python -m playwright install --with-deps
- name: Run your tests
run: pytest
We run our tests on GitHub Actions, across a matrix of 3 platforms (Windows, Linux, macOS) and 3 browsers (Chromium, Firefox, WebKit).
This will start the tests after a GitHub Deployment went into the success
state.
Services like Vercel use this pattern so you can run your end-to-end tests on their deployed environment.
name: Playwright Tests
on:
deployment_status:
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
if: github.event.deployment_status.state == 'success'
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '14.x'
- name: Install dependencies
run: npm ci
- name: Install Playwright
run: npx playwright install --with-deps
- name: Run Playwright tests
run: npm run test:e2e
env:
# This might depend on your test-runner/language binding
PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
We have a pre-built Docker image which can either be used directly, or as a reference to update your existing Docker definitions.
Suggested configuration
- Using
--ipc=host
is also recommended when using Chromium—without it Chromium can run out of memory and crash. Learn more about this option in Docker docs. - Seeing other weird errors when launching Chromium? Try running your container
with
docker run --cap-add=SYS_ADMIN
when developing locally. - Using
--init
Docker flag or dumb-init is recommended to avoid special treatment for processes with PID=1. This is a common reason for zombie processes.
For Windows or macOS agents, no additional configuration required, just install Playwright and run your tests.
For Linux agents, you can use our Docker container with Azure Pipelines support running containerized jobs. Alternatively, you can use Command line tools to install all necessary dependencies.
pool:
vmImage: 'ubuntu-20.04'
container: mcr.microsoft.com/playwright:v1.23.4-focal
steps:
...
Running Playwright on CircleCI requires the following steps:
-
Use the pre-built Docker image in your config like so:
docker: - image: mcr.microsoft.com/playwright:v1.23.4-focal environment: NODE_ENV: development # Needed if playwright is in `devDependencies`
-
If you’re using Playwright through Jest, then you may encounter an error spawning child processes:
[00:00.0] jest args: --e2e --spec --max-workers=36 Error: spawn ENOMEM at ChildProcess.spawn (internal/child_process.js:394:11)
This is likely caused by Jest autodetecting the number of processes on the entire machine (
36
) rather than the number allowed to your container (2
). To fix this, setjest --maxWorkers=2
in your test command.
Jenkins supports Docker agents for pipelines. Use the Playwright Docker image to run tests on Jenkins.
pipeline {
agent { docker { image 'mcr.microsoft.com/playwright:v1.23.4-focal' } }
stages {
stage('e2e-tests') {
steps {
sh 'npm install'
sh 'npm run test'
}
}
}
}
Bitbucket Pipelines can use public Docker images as build environments. To run Playwright tests on Bitbucket, use our public Docker image (see Dockerfile).
image: mcr.microsoft.com/playwright:v1.23.4-focal
To run Playwright tests on GitLab, use our public Docker image (see Dockerfile).
stages:
- test
tests:
stage: test
image: mcr.microsoft.com/playwright:v1.23.4-focal
script:
...
By default, Playwright downloads browser binaries when the Playwright NPM package
is installed. The NPM packages have a postinstall
hook that downloads the browser
binaries. This behavior can be customized with environment variables.
Caching browsers on CI is strictly optional: The postinstall
hooks should
execute and download the browser binaries on every run.
Most CI providers cache the npm-cache
directory (located at $HOME/.npm
). If your CI pipelines caches the node_modules
directory and you run npm install
(instead of npm ci
), the default configuration
will not work. This is because the npm install
step will find the Playwright NPM
package on disk and not execute the postinstall
step.
Travis CI automatically caches
node_modules
if your repo does not have apackage-lock.json
file.
This behavior can be fixed with one of the following approaches:
- Move to caching
$HOME/.npm
or the npm-cache directory. (This is the default behavior in most CI providers.) - Set
PLAYWRIGHT_BROWSERS_PATH=0
as the environment variable before runningnpm install
. This will download the browser binaries in thenode_modules
directory and cache them with the package code. See managing browser binaries. - Use
npm ci
(instead ofnpm install
) which forces a clean install: by removing the existingnode_modules
directory. See npm docs. - Cache the browser binaries, with the steps below.
With the default behavior, Playwright downloads the browser binaries in the following directories:
%USERPROFILE%\AppData\Local\ms-playwright
on Windows~/Library/Caches/ms-playwright
on MacOS~/.cache/ms-playwright
on Linux
To cache the browser downloads between CI runs, cache this location in your CI configuration, against a hash of the Playwright version.
Playwright supports the DEBUG
environment variable to output debug logs during execution. Setting it to pw:browser*
is helpful while debugging Error: Failed to launch browser
errors.
DEBUG=pw:browser* npm run test
DEBUG=pw:browser* pytest
By default, Playwright launches browsers in headless mode. This can be changed by passing a flag when the browser is launched.
// Works across chromium, firefox and webkit
const { chromium } = require('playwright');
const browser = await chromium.launch({ headless: false });
// Works across chromium, firefox and webkit
import com.microsoft.playwright.*;
public class Example {
public static void main(String[] args) {
try (Playwright playwright = Playwright.create()) {
BrowserType chromium = playwright.chromium();
Browser browser = chromium.launch(new BrowserType.LaunchOptions().setHeadless(false));
}
}
}
import asyncio
from playwright.async_api import async_playwright
async def main():
async with async_playwright() as p:
# Works across chromium, firefox and webkit
browser = await p.chromium.launch(headless=False)
asyncio.run(main())
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
# Works across chromium, firefox and webkit
browser = p.chromium.launch(headless=False)
using Microsoft.Playwright;
using var playwright = await Playwright.CreateAsync();
await playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
{
Headless = false
});
On Linux agents, headed execution requires Xvfb to be installed. Our Docker image and GitHub Action have Xvfb pre-installed. To run browsers in headed mode with Xvfb, add xvfb-run
before the Node.js command.
xvfb-run node index.js
xvfb-run python test.py