Skip to content

Commit

Permalink
chore: automate prettier in docs (#7014)
Browse files Browse the repository at this point in the history
Issue: #7012
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
  • Loading branch information
theDavidBarton and mathiasbynens committed Mar 23, 2021
1 parent ff86081 commit c62b02f
Show file tree
Hide file tree
Showing 9 changed files with 760 additions and 287 deletions.
8 changes: 8 additions & 0 deletions .prettierignore
@@ -0,0 +1,8 @@
node_modules/
lib/
third_party/
vendor/

package-lock.json
yarn.lock
package.json
9 changes: 8 additions & 1 deletion CONTRIBUTING.md
@@ -1,3 +1,4 @@
<!-- prettier-ignore-start -->
<!-- gen:toc -->
- [How to Contribute](#how-to-contribute)
* [Contributor License Agreement](#contributor-license-agreement)
Expand All @@ -21,7 +22,7 @@
- [Bisecting upstream changes](#bisecting-upstream-changes)
* [Releasing to npm](#releasing-to-npm)
<!-- gen:stop -->

<!-- prettier-ignore-end -->
# How to Contribute

First of all, thank you for your interest in Puppeteer!
Expand Down Expand Up @@ -164,6 +165,12 @@ To run the documentation linter, use:
npm run doc
```

To format the documentation markdown and its code snippets, use:

```bash
npm run markdownlint-fix
```

## Adding New Dependencies

For all dependencies (both installation and development):
Expand Down
951 changes: 702 additions & 249 deletions docs/api.md

Large diffs are not rendered by default.

29 changes: 18 additions & 11 deletions docs/troubleshooting.md
@@ -1,5 +1,7 @@
# Troubleshooting

<!-- prettier-ignore-start -->

<!-- GEN:toc -->
- [Chrome headless doesn't launch on Windows](#chrome-headless-doesnt-launch-on-windows)
- [Chrome headless doesn't launch on UNIX](#chrome-headless-doesnt-launch-on-unix)
Expand All @@ -21,6 +23,8 @@
- [Code Transpilation Issues](#code-transpilation-issues)
<!-- GEN:stop -->

<!-- prettier-ignore-end -->

## Chrome headless doesn't launch on Windows

Some [chrome policies](https://support.google.com/chrome/a/answer/7532015) might enforce running Chrome/Chromium
Expand Down Expand Up @@ -85,6 +89,7 @@ lsb-release
wget
xdg-utils
```

</details>

<details>
Expand Down Expand Up @@ -118,6 +123,7 @@ After installing dependencies you need to update nss library using this command
```
yum update nss -y
```

</details>

<details>
Expand All @@ -139,7 +145,6 @@ spawn /Users/.../node_modules/puppeteer/.local-chromium/mac-756035/chrome-mac/Ch

This means that the browser was downloaded but failed to be extracted correctly. The most common cause is a bug in Node.js v14.0.0 which broke `extract-zip`, the module Puppeteer uses to extract browser downloads into the right place. The bug was fixed in Node.js v14.1.0, so please make sure you're running that version or higher. Alternatively, if you cannot upgrade, you could downgrade to Node.js v12, but we recommend upgrading when possible.


## Setting Up Chrome Linux Sandbox

In order to protect the host environment from untrusted web content, Chrome uses [multiple layers of sandboxing](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/linux/sandboxing.md). For this to work properly,
Expand All @@ -150,7 +155,9 @@ If you **absolutely trust** the content you open in Chrome, you can launch Chrom
with the `--no-sandbox` argument:

```js
const browser = await puppeteer.launch({args: ['--no-sandbox', '--disable-setuid-sandbox']});
const browser = await puppeteer.launch({
args: ['--no-sandbox', '--disable-setuid-sandbox'],
});
```

> **NOTE**: Running without a sandbox is **strongly discouraged**. Consider configuring a sandbox instead.
Expand Down Expand Up @@ -191,12 +198,12 @@ or `.zshenv`:
export CHROME_DEVEL_SANDBOX=/usr/local/sbin/chrome-devel-sandbox
```


## Running Puppeteer on Travis CI

> 👋 We ran our tests for Puppeteer on Travis CI until v6.0.0 (when we've migrated to GitHub Actions) - see our historical [`.travis.yml` (v5.5.0)](https://github.com/puppeteer/puppeteer/blob/v5.5.0/.travis.yml) for reference.
Tips-n-tricks:

- [xvfb](https://en.wikipedia.org/wiki/Xvfb) service should be launched in order to run Chromium in non-headless mode
- Runs on Xenial Linux on Travis by default
- Runs `npm install` by default
Expand Down Expand Up @@ -366,7 +373,7 @@ longer necessary. Instead, launch the browser with the `--disable-dev-shm-usage`

```js
const browser = await puppeteer.launch({
args: ['--disable-dev-shm-usage']
args: ['--disable-dev-shm-usage'],
});
```

Expand Down Expand Up @@ -422,17 +429,17 @@ If you are using an EC2 instance running amazon-linux in your CI/CD pipeline, an

1. To install Chromium, you have to first enable `amazon-linux-extras` which comes as part of [EPEL (Extra Packages for Enterprise Linux)](https://aws.amazon.com/premiumsupport/knowledge-center/ec2-enable-epel/):

```sh
sudo amazon-linux-extras install epel -y
```
```sh
sudo amazon-linux-extras install epel -y
```

1. Next, install Chromium:

```sh
sudo yum install -y chromium
```
```sh
sudo yum install -y chromium
```

Now Puppeteer can launch Chromium to run your tests. If you do not enable EPEL and if you continue installing chromium as part of `npm install`, Puppeteer cannot launch Chromium due to unavailablity of `libatk-1.0.so.0` and many more packages.
Now Puppeteer can launch Chromium to run your tests. If you do not enable EPEL and if you continue installing chromium as part of `npm install`, Puppeteer cannot launch Chromium due to unavailablity of `libatk-1.0.so.0` and many more packages.

## Code Transpilation Issues

Expand Down
3 changes: 2 additions & 1 deletion examples/README.md
Expand Up @@ -17,7 +17,7 @@ More complex and use case driven examples can be found at [github.com/GoogleChro
## Rendering and web scraping

- [Puppetron](https://github.com/cheeaun/puppetron) - Demo site that shows how to use Puppeteer and Headless Chrome to render pages. Inspired by [GoogleChrome/rendertron](https://github.com/GoogleChrome/rendertron).
- [Thal](https://medium.com/@e_mad_ehsan/getting-started-with-puppeteer-and-chrome-headless-for-web-scrapping-6bf5979dee3e "An article on medium") - Getting started with Puppeteer and Chrome Headless for Web Scraping.
- [Thal](https://medium.com/@e_mad_ehsan/getting-started-with-puppeteer-and-chrome-headless-for-web-scrapping-6bf5979dee3e 'An article on medium') - Getting started with Puppeteer and Chrome Headless for Web Scraping.
- [pupperender](https://github.com/LasaleFamine/pupperender) - Express middleware that checks the User-Agent header of incoming requests, and if it matches one of a configurable set of bots, render the page using Puppeteer. Useful for PWA rendering.
- [headless-chrome-crawler](https://github.com/yujiosaka/headless-chrome-crawler) - Crawler that provides simple APIs to manipulate Headless Chrome and allows you to crawl dynamic websites.
- [puppeteer-examples](https://github.com/checkly/puppeteer-examples) - Puppeteer Headless Chrome examples for real life use cases such as getting useful info from the web pages or common login scenarios.
Expand All @@ -35,4 +35,5 @@ More complex and use case driven examples can be found at [github.com/GoogleChro
- [cucumber-puppeteer-example](https://github.com/mlampedx/cucumber-puppeteer-example) - Example repository demonstrating how to use Puppeeteer and Cucumber for integration testing.

## Services

- [Checkly](https://checklyhq.com) - Monitoring SaaS that uses Puppeteer to check availability and correctness of web pages and apps.
4 changes: 2 additions & 2 deletions experimental/puppeteer-firefox/README.md
Expand Up @@ -21,7 +21,7 @@ Note: When you install puppeteer-firefox, it downloads a [custom-built Firefox](

### Usage

**Example** - navigating to https://example.com and saving a screenshot as *example.png*:
**Example** - navigating to https://example.com and saving a screenshot as `example.png`:

Save file as **example.js**

Expand All @@ -32,7 +32,7 @@ const pptrFirefox = require('puppeteer-firefox');
const browser = await pptrFirefox.launch();
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({path: 'example.png'});
await page.screenshot({ path: 'example.png' });
await browser.close();
})();
```
Expand Down
4 changes: 3 additions & 1 deletion package.json
Expand Up @@ -24,7 +24,9 @@
"eslint": "([ \"$CI\" = true ] && eslint --ext js --ext ts --quiet -f codeframe . || eslint --ext js --ext ts .)",
"eslint-fix": "eslint --ext js --ext ts --fix .",
"commitlint": "commitlint --from=HEAD~1",
"lint": "npm run eslint && npm run build && npm run doc && npm run commitlint",
"markdownlint": "prettier --check **/README.md docs/api.md docs/troubleshooting.md",
"markdownlint-fix": "prettier --write **/README.md docs/api.md docs/troubleshooting.md",
"lint": "npm run eslint && npm run build && npm run doc && npm run commitlint && npm run markdownlint",
"doc": "node utils/doclint/cli.js",
"clean-lib": "rm -rf lib",
"build": "npm run tsc && npm run generate-d-ts",
Expand Down
25 changes: 12 additions & 13 deletions test/README.md
Expand Up @@ -4,19 +4,18 @@ Unit tests in Puppeteer are written using [Mocha] as the test runner and [Expect

## Test state


We have some common setup that runs before each test and is defined in `mocha-utils.js`.

You can use the `getTestState` function to read state. It exposes the following that you can use in your tests. These will be reset/tidied between tests automatically for you:

* `puppeteer`: an instance of the Puppeteer library. This is exactly what you'd get if you ran `require('puppeteer')`.
* `puppeteerPath`: the path to the root source file for Puppeteer.
* `defaultBrowserOptions`: the default options the Puppeteer browser is launched from in test mode, so tests can use them and override if required.
* `server`: a dummy test server instance (see `utils/testserver` for more).
* `httpsServer`: a dummy test server HTTPS instance (see `utils/testserver` for more).
* `isFirefox`: true if running in Firefox.
* `isChrome`: true if running Chromium.
* `isHeadless`: true if the test is in headless mode.
- `puppeteer`: an instance of the Puppeteer library. This is exactly what you'd get if you ran `require('puppeteer')`.
- `puppeteerPath`: the path to the root source file for Puppeteer.
- `defaultBrowserOptions`: the default options the Puppeteer browser is launched from in test mode, so tests can use them and override if required.
- `server`: a dummy test server instance (see `utils/testserver` for more).
- `httpsServer`: a dummy test server HTTPS instance (see `utils/testserver` for more).
- `isFirefox`: true if running in Firefox.
- `isChrome`: true if running Chromium.
- `isHeadless`: true if the test is in headless mode.

If your test needs a browser instance, you can use the `setupTestBrowserHooks()` function which will automatically configure a browser that will be cleaned between each test suite run. You access this via `getTestState()`.

Expand All @@ -33,8 +32,8 @@ There is also `describeChromeOnly` and `itChromeOnly` which will only execute th
There are also tests that assume a normal install flow, with browser binaries ending up in `.local-<browser>`, for example. Such tests are skipped with
`itOnlyRegularInstall` which checks `BINARY` and `PUPPETEER_ALT_INSTALL` environment variables.

[Mocha]: https://mochajs.org/
[Expect]: https://www.npmjs.com/package/expect
[mocha]: https://mochajs.org/
[expect]: https://www.npmjs.com/package/expect

## Running tests

Expand All @@ -46,7 +45,7 @@ Despite being named 'unit', these are integration tests, making sure public API
npm run unit
```

- __Important__: don't forget to first run TypeScript if you're testing local changes:
- **Important**: don't forget to first run TypeScript if you're testing local changes:

```bash
npm run tsc && npm run unit
Expand All @@ -63,7 +62,7 @@ npm run tsc && npm run unit
});
```

- To disable a specific test, substitute the `it` with `xit` (mnemonic rule: '*cross it*'):
- To disable a specific test, substitute the `it` with `xit` (mnemonic rule: '_cross it_'):

```js
...
Expand Down
14 changes: 5 additions & 9 deletions utils/doclint/README.md
@@ -1,18 +1,14 @@
# DocLint

**Doclint** is a small program that lints Puppeteer's documentation against
Puppeteer's source code.
**Doclint** is a small program that lints Puppeteer's documentation against Puppeteer's source code.

Doclint works in a few steps:

1. Read sources in `lib/` folder, parse AST trees and extract public API
- note that we run DocLint on the outputted JavaScript in `lib/` rather than the source code in `src/`. We will do this until we have migrated `src/` to be exclusively TypeScript and then we can update DocLint to support TypeScript.
2. Read sources in `docs/` folder, render markdown to HTML, use puppeteer to traverse the HTML
and extract described API
3. Compare one API to another
1. Read sources in `lib/` folder, parse AST trees and extract public API. Note that we run DocLint on the outputted JavaScript in `lib/` rather than the source code in `src/`. We will do this until we have migrated `src/` to be exclusively TypeScript and then we can update DocLint to support TypeScript.
2. Read sources in `docs/` folder, render markdown to HTML, use puppeteer to traverse the HTML and extract described API.
3. Compare one API to another.

Doclint is also responsible for general markdown checks, most notably for the table of contents
relevancy.
Doclint is also responsible for general markdown checks, most notably for the table of contents relevancy.

## Running

Expand Down

0 comments on commit c62b02f

Please sign in to comment.