node-sass ignores proxy settings when trying to download binaries

[npiguet]

I'm in an environment where I am required to run a local proxy, and I am unable to install node-sass. The output of node-sass looks like:

[INFO] > node-sass@3.0.0-alpha.0 install /home/user/workspace/src/main/frontend/node_modules/node-sass
[INFO] > node scripts/install.js
[INFO] 
[ERROR] tunneling socket could not be established, cause=read ECONNRESET

I've tried setting the proxy in multiple ways, but none of them seem to work:

through environment variables:

$ echo $http_proxy , $https_proxy
http://127.0.0.1:3129/ , https://127.0.0.1:3129/

through .npmrc

https-proxy=https://127.0.0.1:3129/
http-proxy=http://127.0.0.1:3129/
proxy=http://127.0.0.1:3129/

through command line arguments

npm install --https-proxy=https://127.0.0.1:3129/ --proxy=http://localhost:3129

None of this works. I know for sure that the proxy not used because I don't see any requests to https://github.com/sass/node-sass/releases/download//v..... anywhere in the logs of my proxy.

Note that all of the other modules I need are installed correctly by npm, that npm does use the proxy and that I can see all of its requests my proxy logs

[snip lots of lines]
cntlm[1497]: 127.0.0.1 GET http://registry.npmjs.org/fstream
cntlm[1497]: 127.0.0.1 GET http://registry.npmjs.org/which
cntlm[1497]: 127.0.0.1 GET http://registry.npmjs.org/request/-/request-2.51.0.tgz
cntlm[1497]: 127.0.0.1 GET http://registry.npmjs.org/semver/-/semver-4.2.2.tgz
cntlm[1497]: 127.0.0.1 GET http://registry.npmjs.org/npmlog/-/npmlog-1.0.0.tgz
[snip lots of lines]

I'm on ubuntu 14.04, and using node v0.12.0, npmVersion 2.5.1 (via frontend-maven-plugin)

Also note that the messages above are for version 3.0.0-alpha.0, but I get the same error with version 2.0.1 and 3.0.0-pre

[am11]

Does truncating the trailing slash in proxy URL help?

http://127.0.0.1:3129/ , https://127.0.0.1:3129/

to

http://127.0.0.1:3129 , https://127.0.0.1:3129

[npiguet]

No, this has no effect.

[npiguet]

On extra detail that might be relevant: this local proxy actually has it own organization-issued (and self signed) root certificate authority that is used to override the certificates provided by all HTTPS websites.

This means that all HTTPS websites will appear to be signed by this root CA. This root CA is installed in the OS wide trust store, but I'm not sure what trust store is used by node-sass, the request module or npm.

[am11]

Please see this comment: #705 (comment) and post the output. Also you can go through that thread and the related in issue tracker to find some pointers.

[npiguet]

OK, so after some more investigation, it turns out that the problem is caused by the fact that a simple HTTPS proxy is not supported for HTTPS requests.

It works if https_proxy is set to http://127.0.0.1:3129
It fails if https_proxy is set to https://127.0.0.1:3129

This is the complete opposite of all the other applications on my system (including browsers, curl, wget, maven, eclipse, IntelliJ, apt, etc...). The way this works now is really, really counter intuitive, and probably needs to be fixed.

[npiguet]

I've been experimenting a bit more with the install.js, but it seems that none of the options offered by the request module will work with my proxy if I specify its URL with the https protocol. I tried a bunch on combinations, but they all end up in various flavors of ECONNRESET or Socket hang up.

A fix that could make it work for many people would be to try to download the binary using the https-proxy first, and if it fails try using the http-proxy.

This all seems to be due to the HTTP tunneling used by the request module when using proxies. This apparently only works when the proxy is contacted using HTTP (at least with the default settings of cntlm). For some reason, setting tunnel=false doesn't solve the problem and still leads to an ECONNRESET error using https as proxy url, and a Socket hang up error using http as proxy url.

[fvsch]

I struggled with a similar issue and used the same "fix": set npm's https-proxy to http://ourproxy/ instead of https://ourproxy/. Not sure if that meant the request went out as HTTP.

[npiguet]

The problem for me, is that our build script must work across all locations of the organization, and some locations don't use a proxy or use a different one. Therefore setting the proxy URL in the build script is a no go for us. The proxy should be read from the system which is apparently what happens, because the proxy URL is correctly picked up when I don't explicitly specify it.

Trying the download in a couple of different ways could be enough to fix the problem for me (and apparently also for fvsch).

[am11]

Thanks for sharing your thoughts.

We can definitely retry downloading with all available proxies, but IMO, it is not the efficient solution of this problem in general.

As you witnessed, this is either an issue with request or it's a bug in underlying node.js/io.js network layer.

Looking into request issue tracker, I found couple of related issues, such as: request/request#367, request/request#1382 and request/request#1357.

(solution from request/request#1382, which we can handle in our code)

Does adding the following line at install.js#L27 help:

options.headers = {'User-Agent': 'foo'}

?

I am not behind proxy server, so unfortunately I cannot confirm it.

Also, I am all for fixing this issue in our code or where it is originating from (request/node.js) but offload the solution to consumer space.

[npiguet]

The User-Agent thing is not related to this proxy issue at all. That issue was because the target URL (the W3C validator) was throwing an error when the User-Agent is not set. If you google the error message, all hits are related directly to the W3C validator. Therefore whatever is described in request/request#1382 does not apply to this issue.

request/request#1357 doesn't contain enough information to be useful.

request/request#367 seems to be related to self signed certificates, which in my case could apply (I need to do some further checks)

[am11]

@npiguet, have you considered opening this issue with request, detailing your use-case and setup?

[npiguet]

After more debugging with wireshark and cntlm's debug mode, it turns out that the proxy is indeed being called, but is apparently called with some illegal characters causing it to fail. So it seems that ultimately, the issue is with the request module.

However, this install problem seems to be common enough to require a workaround in the node-sass installer.

[am11]

Thanks for the follow up @npiguet. Adding this to v3 milestone.
(this turned out to be a feature request to try all set of proxy available returned by configuration)

Feel free to send a PR, I won't have time to visit this until after next beta.

[saper]

@npiguet Can you post (privately here is my PGP key if you need) your wireshark dumps?

I have configured a litte proxy for myself and I am using

env http_proxy=http://mylittleproxy:9999 npm install

and it works.

Please keep in mind that SSL proxying works also via plain HTTP (it uses CONNECT HTTP method), so you probably need only env http_proxy=http://yourproxy:3129 npm install. You might need to check your local policy for a list of allowed CONNECT destinations and ports.

As far as NPM is concerned, you can also set npm proxy by setting environment variable NPM_CONFIG_PROXY or in .npmrc files. This setting will also be used by the node-sass installation script (even when not using npm install).

[emrecamasuvi]

thnx @npiguet
this solution resolved my proxy problem it #768 (comment)

[xzyfer]

If I'm understanding correctly this is not a node-sass issue. Closing this until there is clear action we can take.