Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(iroh-net)!: Implement http proxy support #2298

Merged
merged 14 commits into from
May 22, 2024
Merged

Conversation

dignifiedquire
Copy link
Contributor

@dignifiedquire dignifiedquire commented May 15, 2024

Headers are based on how curl and env variables onreqwest.

Setting any of

  • HTTP_PROXY
  • http_proxy
  • HTTPS_PROXY
  • https_proxy

will make all relay code use these to proxy outgoing connections.

Closes #2295

Breaking Changes

  • Added iroh_net::endpoint::Builder::proxy_url
  • Added iroh_net::endpoint::Builder::proxy_from_env
  • Added iroh_net::relay::http::ClientError::Proxy enum variant

TODOs

  •  config & parsing env variables
  •  the todos in the code
  • https proxy
  • testing: tested manually on two machines using squid

@dignifiedquire dignifiedquire changed the title [WIP] start implementing proxy support iroh-net: Implement proxy support May 16, 2024
@dignifiedquire dignifiedquire force-pushed the feat-net-proxy branch 2 times, most recently from e4226aa to dca6b0b Compare May 16, 2024 17:13
@dignifiedquire dignifiedquire marked this pull request as ready for review May 16, 2024 17:13
@dignifiedquire dignifiedquire changed the title iroh-net: Implement proxy support feat(iroh-net): Implement http proxy support May 16, 2024
@dignifiedquire dignifiedquire changed the title feat(iroh-net): Implement http proxy support feat(iroh-net)!: Implement http proxy support May 16, 2024
@link2xt
Copy link
Contributor

link2xt commented May 16, 2024

Is it a good idea for a library to read environment variables? I think application should do this, there are more variables like NO_PROXY and at least in GNOME proxy is expected to be controlled via org.gnome.system.proxy. On other platforms proxy may be controlled differently as well.

@dignifiedquire
Copy link
Contributor Author

Is it a good idea for a library to read environment variables?

reqwest is already used, and does exactly this. They do cover a wider variety of supported variables, like NO_PROXY and system specific configurations which we can add over time, but for now this allows existing users who need this to work with iroh, where they can't currently

@zicklag
Copy link

zicklag commented May 16, 2024

Environment variables for proxy configuration are extremely common. curl, wget, apt and more support the environment variables. Almost every tool that supports proxies supports it through environment variables, especially if it has a CLI.

Fundamentally I think it makes sense because it is an environment configuration, that is the same for every tool on the machine, in almost every case, and it doesn't seem valuable to make the user comprehend separate configurations for every app they use that needs to access the network if it isn't necessary.

@link2xt
Copy link
Contributor

link2xt commented May 17, 2024

Environment variable configuration is fine, but I think it should not be done by the libraries. If every library that does networking implements proxy variable parsing, then you will get slightly different handling of these environment variables within a single application: some libraries parse only uppercase variables, some handle NO_PROXY and some don't etc.

Environment variables also have a downside that they cannot be reconfigured in runtime, so in Android applications you probably want to have some hooks that detect network changes, on desktop Linux you want to hook into NetworkManager changes because proxy may be needed in a corporate VPN and not even accessible outside this VPN. This kind of hooks/reconfigurations better happens on the application side because Rust libraries may not even be able to access necessary Java APIs on Android.

Parsing environment variables in reqwest is a bad decision IMO, somewhat similar to libraries that log error messages to stderr.

@zicklag
Copy link

zicklag commented May 17, 2024

I see your point.

Maybe we should have a Proxy config struct that could be passed into the ClientBuilder or something like that. We could have a Proxy::from_env() function, and you could have proxies disabled by default in the library.

Then the Iroh CLI will configure the client with the Proxy::from_env().

It would actually be handy to have the Proxy struct, and the environment variable parsing logic in a separate crate so that it could be re-used.

@dignifiedquire
Copy link
Contributor Author

the environment now needs to be read explicitly in iroh-net, but iroh will by default read it

@dignifiedquire dignifiedquire added this to the v0.17.0 milestone May 22, 2024
Copy link
Contributor

@flub flub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the approach of the builder in Endpoint. Only minor note I have is that we now have two pin-project crates...

iroh-net/src/relay/http/client.rs Outdated Show resolved Hide resolved
iroh-net/Cargo.toml Outdated Show resolved Hide resolved
@dignifiedquire dignifiedquire added this pull request to the merge queue May 22, 2024
@flub flub removed this pull request from the merge queue due to a manual request May 22, 2024
@dignifiedquire dignifiedquire added this pull request to the merge queue May 22, 2024
Merged via the queue into main with commit 6d1a6dd May 22, 2024
22 of 23 checks passed
@zicklag
Copy link

zicklag commented May 22, 2024

Thanks a bunch for this work. ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Iroh fails when running through an outbound HTTPS proxy
4 participants