Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can this client (or any?) make multiple concurrent outbound requests? #335

Open
ahungry opened this issue Apr 5, 2024 · 1 comment
Open

Comments

@ahungry
Copy link

ahungry commented Apr 5, 2024

Hi, a bit new to Erlang - is it possible to start, say, 3 outbound HTTP requests simultaneously, and collect the results from the caller (such that if each remote endpoint took ~2 seconds to reply, the total duration on the calling side would be ~2 seconds, and not 6)?

I've come across erpc.multicall and rpc.pmap in the stdlib, but I'm not sure either of those fit this use case, so curious if this would be something supported in the http client(s) out there themselves? (though search matches for "concurren" seem sparse).

I did see Elixir has a new Task.await_many() which seems similar (basically, JS Promise.all() equivalent).

Sorry for the slightly off topic question, but TIA if anyone can answer.

@essen
Copy link
Member

essen commented Apr 5, 2024

Yes and no.

  • Using HTTP/1.1, you can use Gun to do concurrent requests but you need to open 3 separate connections. This is because HTTP/1.1 is sequential. Browsers have used multiple connections for a long time because of that.
  • Using HTTP/2 (and, when it becomes available, HTTP/3), you can use a single Gun connection to do concurrent requests.

Gun is asynchronous so you can do the requests all at once and gather the results afterwards:

StreamRef1 = gun:get(ConnPid, "/one"),
StreamRef2 = gun:get(ConnPid, "/two"),
StreamRef3 = gun:get(ConnPid, "/three"),
gather_results(StreamRef1, StreamRef2, StreamRef3)

Your gather_results function (which could be your main receive function) can receive the responses asynchronously if necessary. The responses and data is sent as messages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants