curl on docs.github.com results in HTTP 403 with "The request is blocked"

[mikouaj]

Code of Conduct

• I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?

I assume that all pages from docs.github.com are affected, tested on:

• https://help.github.com/articles/about-pull-requests/
• https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests

Check Gist with verbose output of curl.

What changes are you suggesting?

The content on docs.github.com should be widely accessible.
Many CONTRIBUTING.md files contain link to the below docs.github.com page:

• https://help.github.com/articles/about-pull-requests/

We use Markdown links checker that follows the links in markdown files to verify them. It fails on 'docs.github.com' links due to HTTP 403 response code.

Additional information

No response

[welcome]

Thanks for opening this issue. A GitHub docs team member should be by to give feedback soon. In the meantime, please check out the contributing guidelines.

[ramyaparimi]

@mikouaj
Thanks so much for opening an issue! I'll triage this for the team to review 👀

[nikitar]

Same issue here. It seems to depend on your ip (I get different results on & off vpn)

[janiceilene]

@mikouaj Thanks so much for opening an issue to let us know what you're seeing and I'm sorry you're seeing issues with the docs site! We're tracking this internally, so I'm going to close this now 💛

[praveenkumar]

Not sure why this issue is closed until it is fixed (even tracked internally), I am also experiencing the same for docs.github.com

$ curl -I https://docs.github.com
HTTP/2 403 
x-azure-ref: 0HTJqYgAAAAD1pEFVgbNxSa5wCE6sfvbYRFhCMzBFREdFMDIwNwA1OTZkNzhhMi1jYTVmLTQ3OWQtYmNkYy0wODM1ODMzMTc0YjI=
accept-ranges: bytes
date: Thu, 28 Apr 2022 06:20:13 GMT
via: 1.1 varnish
x-served-by: cache-maa10222-MAA
x-cache: MISS
x-cache-hits: 0
x-timer: S1651126813.090800,VS0,VE198
strict-transport-security: max-age=31557600

but for github.com it is working

$ curl -I https://github.com
HTTP/2 200 
server: GitHub.com
date: Thu, 28 Apr 2022 06:20:05 GMT
content-type: text/html; charset=utf-8
vary: X-PJAX, X-PJAX-Container, Accept-Language, Accept-Encoding, Accept, X-Requested-With
permissions-policy: interest-cohort=()
content-language: en-US
etag: W/"905ccee40f4cb9273cc299723eb39e06"
cache-control: max-age=0, private, must-revalidate

[cfergeau]

curl --compressed -I https://docs.github.com works though (this adds a accept-encoding: deflate, gzip, br header to the request)

[larshp]

same issue, when checking links with https://github.com/gaurav-nelson/asciidoc-link-check

[sdruskat]

@janiceilene and colleagues, were you able to find the cause of this? I'm getting the same issue using https://github.com/gaurav-nelson/github-action-markdown-link-check (gaurav-nelson/github-action-markdown-link-check#143), but am not convinced that this isn't an issue with the link checker action. Would be good to know of any leads. Thanks!

[riccardoporreca]

@sdruskat, this seems known for tcort/markdown-link-check, which /gaurav-nelson/github-action-markdown-link-check relies on: See issue tcort/markdown-link-check#201 linked above, which is in turns already referencing a different issue gaurav-nelson/github-action-markdown-link-check#136. It seems solutions/workarounds should be possible