pageres swamping server with requests #416
Comments
|
I just tried on a local web-server and I cannot reproduce this. It correctly does a single request. Pageres doesn't have any retry logic, so either there's a bug in Puppeteer (headless Chrome) or the server is misconfigured. |
|
Thanks for the reply. I witnessed that on two servers. Let me know if you're interested in a URL to check it out -- I'll just have to send it to you privately.
…On Wed, Jun 23, 2021 at 05:03:56AM -0700, Sindre Sorhus wrote:
I just tried on a local web-server and I cannot reproduce this. It correctly does a single request. Pageres doesn't have any retry logic, so either there's a bug in Puppeteer or the server is misconfigured.
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
#416 (comment)
|
|
Try the latest version: https://github.com/sindresorhus/pageres/releases/tag/v7.0.0 It may have resolved the issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Fresh pageres-cli installed today, it hammers the server with repeat requests before ultimately stopping and successfully producing the screenshot.
I noticed the issue on a production server when the generated screenshots were all errors with HTTP code 429 ("too many requests").
$ pageres http://www.mytestsite.com
Server logs, see timestamps (goes much faster on production server)
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:55 +0300] "GET / HTTP/1.1" 200 5736 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.045 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:55 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.042 "- : 0.040"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.046 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.047 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.148 "- : 0.148"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.049 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.046 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:56 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.044 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.045 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.061 "- : 0.060"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.055 "- : 0.056"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.044 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.064 "- : 0.064"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:57 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.052 "- : 0.052"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.052 "- : 0.052"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.054 "- : 0.052"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.052 "- : 0.052"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.045 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.043 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.042 "- : 0.040"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.051 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:58 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.048 "- : 0.048"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.039 "- : 0.040"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.056 "- : 0.056"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.055 "- : 0.052"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.044 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.042 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:08:59 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.042 "- : 0.044"
www.mytestsite.com "127.0.0.1" - - [21/Jun/2021:14:09:00 +0300] "GET / HTTP/1.1" 304 502 "http://www.mytestsite.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/88.0.4298.0 Safari/537.36" 0.063 "- : 0.064"
[...]
The text was updated successfully, but these errors were encountered: