Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perform HEAD request before GET #25

Open
Kikobeats opened this issue May 1, 2018 · 7 comments
Open

Perform HEAD request before GET #25

Kikobeats opened this issue May 1, 2018 · 7 comments

Comments

@Kikobeats
Copy link

I noted the library is doing a GET request to determinate if the url is reachable:

https://github.com/sindresorhus/is-reachable/blob/master/index.js#L16

Normally you can use HEAD request for doing that, but some sites disable perform this request over his sites.

I found this user case in the past and my solution was to try to perform a HEAD request first, fallback into GET if is not possible:

https://github.com/urlint/url-code-status/blob/master/index.js#L16

Because HEAD is lightweight GET, make sense if you want to accept a PR improving that?

@sindresorhus
Copy link
Owner

Sure, makes sense. PR welcome.

@silverwind
Copy link
Collaborator

Keep in mind that not all HTTP servers will support HEAD requests. If you want to speed up things, I suggest switching to core {http,https}.get and opt to not parse the response body.

@Kikobeats
Copy link
Author

That's a good point!

HEAD and GET will be launched on parallel and the response will be the first request to be resolved.

Because got is using http/https internally, we can avoid parsing body as you suggest 🙂

@silverwind
Copy link
Collaborator

silverwind commented May 4, 2018

HEAD and GET will be launched on parallel

That seems pretty wasteful of resources to me, not sure I like it.

Because got is using http/https internally, we can avoid parsing body as you suggest

Doesn't got always parse the body unconditionally?

@Kikobeats
Copy link
Author

That seems pretty wasteful of resources to me, not sure I like it.

I understand your point and I did tests based on this premise. Sometimes HEAD response is fastest than GET. In the case HEAD is not supported, it doesn't be done. Need to test it again disabling parse body response.

Doesn't got always parse the body unconditionally?

got accepts http request as options. I don't know how exactly specify about doesn't parse response (maybe @sindresorhus help here could be useful!). The point why I want to use got is to have the rest of the library benefits, like follow redirects

@silverwind
Copy link
Collaborator

silverwind commented May 4, 2018

The point why I want to use got is to have the rest of the library benefits, like follow redirects

We don't actually need to follow redirects in all cases. Part of the purpose of the checkRedirection function is to make an attempt at detecting captive portals that are realized through HTTP redirects. For this case, we should not follow a redirect (as detection can be done based on the response to the first request alone), for all others we shoud.

So in summary:

  1. Issue node http(s) GET
  2. If response is a redirect to a router ip, return false
  3. If response code is not 3xx, return true, otherwise issue more GETs and possibly also HEADs, following redirects.

Also keep in mind that got does not support proxies. I've been using request with request-promise-native lately because of that, maybe you should do that here too.

@Kikobeats
Copy link
Author

I noted got@9 accepts a null as encoding value for avoiding parse the response data: https://github.com/sindresorhus/got#encoding

Could this change in the codebase make an improvement as you suggest? #25 (comment)?

btw, this library needs to upgrade got dependency

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants