pick up http_proxy and https_proxy environment variables

[toymil]

💡 Feature description

Pick up http_proxy and https_proxy environment variables for all the download tasks (like downloading chromedriver when executing wasm-pack test --chrome --headless for the first time).

This will prevent frustrating network problems like #605 and #1345 , since there is currently no way to set proxy for wasm-pack.

Solution

From what I can tell from the source code, wasm-pack both directly and indirectly uses ureq for all the download tasks, indirectly through binary-install.

• impl a feature flag to enable auto proxy pick up by default in ureq
  • wait for version release
• update binary-install's ureq dep, and enable (or pass through since binary-install is a lib) related feature flag
  • wait for version release
• update wasm-pack's binary-install and ureq dep, enable related feature flag

[algesten]

I believe this is done. It's released in ureq 2.8.0 (and we're on 2.9.0 now).

[toymil]

I did the following:

• clone ureq, checkout 2.9.0
  • modified to debug print proxy settings each time a request is made, to confirm if proxy envvars have been picked up or not.
• clone binary-install, checkout v0.2.0
  • change Cargo.toml to use local modified ureq
• clone wasm-pack, checkout v0.12.1
  • change Cargo.toml to use local modified binary-install and ureq
• install this local version of wasm-pack for tests

and the tests are negative, none of the network requests picked up proxy envvars...

Will do a bit more digging to see if I missed anything.

[toymil]

I see that wasm-pack and binary-install all calls ureq::get(..) to download, which initializes an agent with the default setting AgentBuilder::new().build(), and by default try_proxy_from_env of AgentBuilder is set to false, so it ended up not using the proxy environment variables.

https://github.com/algesten/ureq/blob/260213b3e2a55e40f7132374034520455d497789/src/agent.rs#L252-L275

https://github.com/algesten/ureq/blob/260213b3e2a55e40f7132374034520455d497789/src/agent.rs#L323-L332

[algesten]

Yes. To use this, create an Agent, enable the environment option, and make the request from the agent.

[toymil]

Change try_proxy_from_env default to true made everything work. Here is what I modified:

• clone ureq, checkout 2.9.0
  • modified to debug print proxy settings each time a request is made, to confirm if proxy env vars have been picked up or not.
  • modified AgentBuilder::new() to enable try_proxy_from_env by default
• clone binary-install, checkout v0.2.0
  • change Cargo.toml to use local modified ureq
• clone wasm-pack, checkout v0.12.1
  • change Cargo.toml to use local modified binary-install and ureq
• install this local version of wasm-pack for tests

And wasm-pack successfully picked up my proxy env vars.

---

The most simple scenario would be for ureq to default try_proxy_from_env to true, then the changes needed here are:

• wasm-pack and binary-install both update the ureq dependency.
• optionally enable socks-proxy feature flag for ureq, so socks proxies in http(s)_proxy env var are also supported.

If not, then we need to manually construct Agent and call AgentBuilder::try_proxy_from_env(mut self, do_try: bool) to enable the behavior each time we need to download. Since there is multiple places in the code where ureq are called, this would be tricky.

[toymil]

@algesten may I ask why try_proxy_from_env is default to false? The comment wrote:

The default is false, i.e. not detecting proxy from env since this is a potential security risk.

But most of the tooling I use does do this by default. So why not enable it by default and provide an option for library users to disable instead?

[toymil]

But most of the tooling I use does do this by default. So why not enable it by default and provide an option for library users to disable instead?

I understand that it might always been the case for network libs to disable this behavior by default, and the reason why "most of the tooling I use does do this by default" is because the tooling devs manually enabled it.

Is this the case?

[algesten]

The motivation is hidden in the PR. I should probably bring it out in the docs. https://github.com/algesten/ureq/pull/649/files#r1290992185