Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: https-proxy-agent@^5.0.0 to fix node@15 ERR_INVALID_PROTOCOL #6555

Merged
merged 2 commits into from Nov 26, 2020
Merged

fix: https-proxy-agent@^5.0.0 to fix node@15 ERR_INVALID_PROTOCOL #6555

merged 2 commits into from Nov 26, 2020

Conversation

dr-js
Copy link
Contributor

@dr-js dr-js commented Oct 29, 2020

With nodejs@15.0.1, install puppeteer with https_proxy=http://*/ set will cause error like:

> puppeteer@5.4.1 install /home/dr/Git/dr-js/node_modules/puppeteer
> node install.js

ERROR: Failed to set up Chromium r809590! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.
TypeError [ERR_INVALID_PROTOCOL]: Protocol "https:" not supported. Expected "http:"
    at new NodeError (node:internal/errors:258:15)
    at new ClientRequest (node:_http_client:155:11)
    at Object.request (node:https:313:10)
    at httpRequest (/home/dr/Git/dr-js/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:488:17)
    at downloadFile (/home/dr/Git/dr-js/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:357:21)
    at BrowserFetcher.download (/home/dr/Git/dr-js/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:239:19)
    at async downloadBrowser (/home/dr/Git/dr-js/node_modules/puppeteer/lib/cjs/puppeteer/node/install.js:48:5) {
  code: 'ERR_INVALID_PROTOCOL'
}

The related issue is at TooTallNate/node-agent-base#47,
from package agent-base under https-proxy-agent

And the https-proxy-agent major version bump is for Refactor to TypeScript: TooTallNate/proxy-agents@4.0.0...5.0.0


Also there's an older "https-proxy-agent": "^2.2.1", under experimental/puppeteer-firefox/package.json, I did not change that since it's marked experimental.

And npm@6 also uses https-proxy-agent but didn't hit the protocol auto detect bug, because it uses http-proxy-agent separately for http_proxy in make-fetch-happen/agent.js#L185-L199 (but npm@7 did hit the bug npm/cli#2003)


And a simple package test: (with nodejs@15.0.1 & npm@7.0.5)

nano package.json # paste original file
npm i --production --package-lock-only --ignore-scripts # just get the `package-lock.json`
mv package-lock.json package-lock.json-org # rename for next run

nano package.json # change to `"https-proxy-agent": "^5.0.0",`
npm i --production --package-lock-only --ignore-scripts

diff package-lock.json-org package-lock.json

and the diff result:

16c16
<         "https-proxy-agent": "^4.0.0",
---
>         "https-proxy-agent": "^5.0.0",
1599,1601c1599,1604
<       "version": "5.1.1",
<       "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-5.1.1.tgz",
<       "integrity": "sha512-TMeqbNl2fMW0nMjTEPOwe3J/PRFP4vqeoNuQMG0HlMrtm5QxKqdvAkZ1pRBQ/ulIyDD5Yq0nJ7YbdD8ey0TO3g==",
---
>       "version": "6.0.2",
>       "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-6.0.2.tgz",
>       "integrity": "sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==",
>       "dependencies": {
>         "debug": "4"
>       },
5296,5298c5299,5301
<       "version": "4.0.0",
<       "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-4.0.0.tgz",
<       "integrity": "sha512-zoDhWrkR3of1l9QAL8/scJZyLu8j/gBkcwcaQOZh7Gyh/+uJQzGVETdgT30akuwkpL8HTRfssqI3BZuV18teDg==",
---
>       "version": "5.0.0",
>       "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-5.0.0.tgz",
>       "integrity": "sha512-EkYm5BcKUGiduxzSt3Eppko+PiNWNEpa4ySk9vTC6wDsQJW9rHSa+UhGNJoRYp7bz6Ht1eaRIa6QaJqO5rCFbA==",
5300c5303
<         "agent-base": "5",
---
>         "agent-base": "6",
5304c5307
<         "node": ">= 6.0.0"
---
>         "node": ">= 6"
7994a7998,8006
>     "node_modules/puppeteer-core/node_modules/agent-base": {
>       "version": "5.1.1",
>       "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-5.1.1.tgz",
>       "integrity": "sha512-TMeqbNl2fMW0nMjTEPOwe3J/PRFP4vqeoNuQMG0HlMrtm5QxKqdvAkZ1pRBQ/ulIyDD5Yq0nJ7YbdD8ey0TO3g==",
>       "dev": true,
>       "engines": {
>         "node": ">= 6.0.0"
>       }
>     },
8000a8013,8025
>     "node_modules/puppeteer-core/node_modules/https-proxy-agent": {
>       "version": "4.0.0",
>       "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-4.0.0.tgz",
>       "integrity": "sha512-zoDhWrkR3of1l9QAL8/scJZyLu8j/gBkcwcaQOZh7Gyh/+uJQzGVETdgT30akuwkpL8HTRfssqI3BZuV18teDg==",
>       "dev": true,
>       "dependencies": {
>         "agent-base": "5",
>         "debug": "4"
>       },
>       "engines": {
>         "node": ">= 6.0.0"
>       }
>     },
11670,11672c11695,11700
<       "version": "5.1.1",
<       "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-5.1.1.tgz",
<       "integrity": "sha512-TMeqbNl2fMW0nMjTEPOwe3J/PRFP4vqeoNuQMG0HlMrtm5QxKqdvAkZ1pRBQ/ulIyDD5Yq0nJ7YbdD8ey0TO3g=="
---
>       "version": "6.0.2",
>       "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-6.0.2.tgz",
>       "integrity": "sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==",
>       "requires": {
>         "debug": "4"
>       }
14540,14542c14568,14570
<       "version": "4.0.0",
<       "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-4.0.0.tgz",
<       "integrity": "sha512-zoDhWrkR3of1l9QAL8/scJZyLu8j/gBkcwcaQOZh7Gyh/+uJQzGVETdgT30akuwkpL8HTRfssqI3BZuV18teDg==",
---
>       "version": "5.0.0",
>       "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-5.0.0.tgz",
>       "integrity": "sha512-EkYm5BcKUGiduxzSt3Eppko+PiNWNEpa4ySk9vTC6wDsQJW9rHSa+UhGNJoRYp7bz6Ht1eaRIa6QaJqO5rCFbA==",
14544c14572
<         "agent-base": "5",
---
>         "agent-base": "6",
16649a16678,16683
>         "agent-base": {
>           "version": "5.1.1",
>           "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-5.1.1.tgz",
>           "integrity": "sha512-TMeqbNl2fMW0nMjTEPOwe3J/PRFP4vqeoNuQMG0HlMrtm5QxKqdvAkZ1pRBQ/ulIyDD5Yq0nJ7YbdD8ey0TO3g==",
>           "dev": true
>         },
16654a16689,16698
>         },
>         "https-proxy-agent": {
>           "version": "4.0.0",
>           "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-4.0.0.tgz",
>           "integrity": "sha512-zoDhWrkR3of1l9QAL8/scJZyLu8j/gBkcwcaQOZh7Gyh/+uJQzGVETdgT30akuwkpL8HTRfssqI3BZuV18teDg==",
>           "dev": true,
>           "requires": {
>             "agent-base": "5",
>             "debug": "4"
>           }

With `nodejs@15.0.1`, install puppeteer with `https_proxy` set will cause error like:
```
> puppeteer@5.4.1 install /home/dr/Git/dr-js/node_modules/puppeteer
> node install.js

ERROR: Failed to set up Chromium r809590! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.
TypeError [ERR_INVALID_PROTOCOL]: Protocol "https:" not supported. Expected "http:"
    at new NodeError (node:internal/errors:258:15)
    at new ClientRequest (node:_http_client:155:11)
    at Object.request (node:https:313:10)
    at httpRequest (/home/dr/Git/dr-js/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:488:17)
    at downloadFile (/home/dr/Git/dr-js/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:357:21)
    at BrowserFetcher.download (/home/dr/Git/dr-js/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:239:19)
    at async downloadBrowser (/home/dr/Git/dr-js/node_modules/puppeteer/lib/cjs/puppeteer/node/install.js:48:5) {
  code: 'ERR_INVALID_PROTOCOL'
}
```

The related issue is at TooTallNate/node-agent-base#47,
  from package `agent-base` under `https-proxy-agent`

And the version bump is for `Refactor to TypeScript`: TooTallNate/proxy-agents@4.0.0...5.0.0
@google-cla
Copy link

google-cla bot commented Oct 29, 2020

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@google-cla google-cla bot added the cla: no label Oct 29, 2020
@dr-js
Copy link
Contributor Author

dr-js commented Oct 29, 2020

@googlebot I signed it!

@google-cla google-cla bot added cla: yes and removed cla: no labels Oct 29, 2020
@dr-js
Copy link
Contributor Author

dr-js commented Oct 30, 2020

Testing the patched .tgz with https_proxy=http://*/ env set do install successfully.

And for me the proxy setting is in ~/.npmrc like:

noproxy=127.0.0.1,localhost # exclude localhost
proxy=http://127.0.0.1:1080 # for http
https-proxy=http://127.0.0.1:1080 # for https

@mathiasbynens mathiasbynens merged commit 3bf5a55 into puppeteer:main Nov 26, 2020
@mathiasbynens
Copy link
Member

Thank you for the patch and the detailed tests + logs in the description!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants