[Bug]: v19.8.0 introduced issue with ignored timeout setting in methods like waitForSelector & gotTo #9927

dekelev · 2023-03-28T11:08:30Z

Bug expectation

Running the gotTo or waitForSelector methods with the timeout option or with a default page timeout that is set to a value higher than 30000 will timeout after 30 seconds regardless of the timeout setting.

Expected behavior is to consider the timeout option or the default page timeout value.

Minimal, reproducible example

Issue does not reproduced with the previous version 19.7.5 and with v19.8.0 it can be reproduce with the following example code:

package.json:

{
  "name": "test-puppeteer-upgrade",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "dependencies": {
    "express": "^4.18.2",
    "puppeteer": "19.8.0",
    "sleep-promise": "^9.1.0"
  }
}

index.html

<!DOCTYPE html>
<html>
<body>
    <h1 id="title" style="display: none">Test</h1>
</body>
<script>
    setTimeout(() => {
        document.getElementById('title').removeAttribute('style')
    }, 35000)
</script>
</html>

index.js

const puppeteer = require('puppeteer');
const express = require('express');
const path = require('path');
const sleep = require('sleep-promise');

const app = express();
const port = process.env.PORT || 8080;

app.get('/', async function (req, res) {
    await sleep(35000)

    res.sendFile(path.join(__dirname, '/index.html'));
});

app.listen(port);

console.log('Server started at http://localhost:' + port);

(async () => {
    const browser = await puppeteer.launch({ headless: false });
    const page = await browser.newPage();

    await page.setDefaultTimeout(180000)

    try {
        await page.goto(`http://localhost:${port}`);
        await page.waitForSelector('#title', { visible: true/*, timeout: 180000*/ })
    } catch (err) {
        console.error(err);
    }

    await browser.close();
})();

This example will get this navigation timeout error after 30 seconds:

Page.navigate timed out.

When removing the sleep before responding with the HTML page, you can test for the waitForSelector issue that will timeout after 30 seconds:

Waiting for selector `#title` failed: Runtime.callFunctionOn timed out.

Error string

Page.navigate timed out.

Puppeteer configuration

{
  headless: false
}

Puppeteer version

19.8.0

Node version

18.12.1

Package manager

npm

Package manager version

8.19.2

Operating system

macOS

The text was updated successfully, but these errors were encountered:

github-actions · 2023-03-28T11:12:34Z

This issue was not reproducible. Please check that your example runs locally and the following:

Ensure the script does not rely on dependencies outside of puppeteer and puppeteer-core.

Ensure the error string is just the error message.

Bad:

Error: something went wrong
  at Object.<anonymous> (/Users/username/repository/script.js:2:1)
  at Module._compile (node:internal/modules/cjs/loader:1159:14)
  at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
  at Module.load (node:internal/modules/cjs/loader:1037:32)
  at Module._load (node:internal/modules/cjs/loader:878:12)
  at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
  at node:internal/main/run_main_module:23:47

Good: Error: something went wrong.

Ensure your configuration file (if applicable) is valid.
If the issue is flaky (does not reproduce all the time), make sure 'Flaky' is checked.
If the issue is not expected to error, make sure to write 'no error'.

Once the above checks are satisfied, please edit your issue with the changes and we will
try to reproduce the bug again.

Analyzer run

OrKoN · 2023-03-28T11:16:25Z

Unfortunately, it is not possible to consider these timeout settings on the protocol client level. I am gonna bump the default timeout in the next release and for now the workaround is to configure the protocolTimeout to be 0 (for no timeout) or to a value that is higher than any timeouts you use.

P.S. I will check if we can plumb the timeouts from the page level to the protocol client.

dekelev · 2023-03-28T11:24:49Z

Thanks @OrKoN

BTW waitForSelector throw timeout after 30 seconds even when the timeout option is set directly (through the method's options object) to a higher value.

I downgraded puppeteer in my project to v19.7.5 for now, since I don't really need the 19.8.0 version specifically and everything seems to work well in v19.7.5.

This PR increases the protocol timeout to be less likely triggered by normal operations but still indicate errors in case of backend issues instead of waiting forever. Unfortunately, the current script evaluation API does not allow configuring the timeouts per operations and it's not possible to change this due to variadic arguments accepted in evaluate and evaluateHandle. We could consider exposing new methods which also accept a timeout but for now let's see if increasing the connection timeout is good enough. Issue #9927

OrKoN · 2023-03-28T11:28:47Z

@dekelev you can configure the protocolTimeout on the launch/connect interface to increase the default protocol timeout (which is 30sec and which was added in 19.8.0). We unlikely to remove the default protocol timeout so if you need a very high failure for your use case, you would need to re-configure the default.

This PR increases the protocol timeout to be less likely triggered by normal operations but still indicate errors in case of backend issues instead of waiting forever. Unfortunately, the current script evaluation API does not allow configuring the timeouts per operations and it's not possible to change this due to variadic arguments accepted in evaluate and evaluateHandle. We could consider exposing new methods which also accept a timeout but for now let's see if increasing the connection timeout is good enough. Issue #9927

dekelev · 2023-03-28T11:59:58Z

I can confirm that setting the v19.8.0 puppeteer launch option, protocolTimeout, to the highest timeout that I use in my project solved the issue.

This PR increases the protocol timeout to be less likely triggered by normal operations but still indicate errors in case of backend issues instead of waiting forever. Unfortunately, the current script evaluation API does not allow configuring the timeouts per operations and it's not possible to change this due to variadic arguments accepted in evaluate and evaluateHandle. We could consider exposing new methods which also accept a timeout but for now let's see if increasing the connection timeout is good enough. Issue #9927

Maxim-Mazurok · 2023-04-02T12:11:39Z

I'm having related error: ProtocolError: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed. puppeteer

I don't believe it to be caused by waitForSelector methods, but I do have a very long-running await page.evaluate(...) which perhaps causes the issue.

I think it's second time puppeteer breaks my code with issues related to timeouts, it would be great to have some tests that would prevent this in the future.

Reverting back from 19.8.2 to 19.7.5 resolved the issue. protocolTimeout: 0 on 19.8.2 seems to be working as well.

OrKoN · 2023-04-04T06:29:22Z

The new timeout setting applies to all CDP calls (see #9801 and there were similar feature requests) and waitForSelector calls evaluate internally. We are going to stick with 180s timeout by default with an option to change that.

nirinsanity · 2023-04-26T09:20:35Z

@OrKoN Please update the corresponding doc page to reflect these changes. I spent close to half a day trying to figure out what was wrong with my code, after updating to v19.8.0.

OrKoN · 2023-04-26T09:27:04Z

The doc you linked is not related to this change and it is not configurable on the waitForFunction methods, the timeout applies globally and it is documented in the changelog and here https://pptr.dev/api/puppeteer.browserconnectoptions/#properties

nirinsanity · 2023-04-26T09:35:24Z

@OrKoN Please help me understand what the issue is, as I thought I was facing the same issue as OP was. The code await page.waitForFunction('window.someFlag === true', { timeout: 45000 }) works on v19.7.5 but not on v19.8.0 or higher.

OrKoN · 2023-04-26T09:40:31Z

@nirinsanity please open a bug with a reproduction scripts and a test page specifying all details.

nirinsanity · 2023-04-26T16:39:44Z

Sorry, I would love to do all that procedure, but my company, which uses puppeteer, doesn't pay me nearly enough to do all that. I'll stick to v19.7.5 for now, thanks.

) (#169013) This is a backport PR for the screenshotting protocolTimeout PR bug fix #167335 for the 7.17 branch The screenshotting plugin was not separate from the reporting plugin until after the 8.x release so this PR refactors the 7.17 version to avoid the puppeteer changes (see puppeteer/puppeteer#9927)

dekelev added the bug label Mar 28, 2023

github-actions bot added needs-feedback not-reproducible labels Mar 28, 2023

OrKoN added confirmed and removed needs-feedback not-reproducible labels Mar 28, 2023

OrKoN self-assigned this Mar 28, 2023

OrKoN mentioned this issue Mar 28, 2023

fix: increase the default protocol timeout #9928

Merged

OrKoN mentioned this issue Mar 28, 2023

[Bug]: #9930

Closed

2 tasks

OrKoN closed this as completed Apr 4, 2023

chenjiahan mentioned this issue Jun 13, 2023

test(e2e): fix puppeteer protocol timeout issue web-infra-dev/modern.js#3950

Merged

3 tasks

rshen91 mentioned this issue Oct 16, 2023

[7.17] [Reporting] Add protocolTimeout to screenshotting plugin (#167335) elastic/kibana#169013

Merged

AgnesToulet mentioned this issue Nov 16, 2023

Puppeteer: Set protocolTimeout grafana/grafana-image-renderer#482

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: v19.8.0 introduced issue with ignored timeout setting in methods like waitForSelector & gotTo #9927

[Bug]: v19.8.0 introduced issue with ignored timeout setting in methods like waitForSelector & gotTo #9927

dekelev commented Mar 28, 2023 •

edited

github-actions bot commented Mar 28, 2023

OrKoN commented Mar 28, 2023 •

edited

dekelev commented Mar 28, 2023

OrKoN commented Mar 28, 2023

dekelev commented Mar 28, 2023

Maxim-Mazurok commented Apr 2, 2023 •

edited

OrKoN commented Apr 4, 2023

nirinsanity commented Apr 26, 2023

OrKoN commented Apr 26, 2023

nirinsanity commented Apr 26, 2023 •

edited

OrKoN commented Apr 26, 2023

nirinsanity commented Apr 26, 2023

[Bug]: v19.8.0 introduced issue with ignored timeout setting in methods like waitForSelector & gotTo #9927

[Bug]: v19.8.0 introduced issue with ignored timeout setting in methods like waitForSelector & gotTo #9927

Comments

dekelev commented Mar 28, 2023 • edited

Bug expectation

Minimal, reproducible example

Error string

Puppeteer configuration

Puppeteer version

Node version

Package manager

Package manager version

Operating system

github-actions bot commented Mar 28, 2023

OrKoN commented Mar 28, 2023 • edited

dekelev commented Mar 28, 2023

OrKoN commented Mar 28, 2023

dekelev commented Mar 28, 2023

Maxim-Mazurok commented Apr 2, 2023 • edited

OrKoN commented Apr 4, 2023

nirinsanity commented Apr 26, 2023

OrKoN commented Apr 26, 2023

nirinsanity commented Apr 26, 2023 • edited

OrKoN commented Apr 26, 2023

nirinsanity commented Apr 26, 2023

dekelev commented Mar 28, 2023 •

edited

OrKoN commented Mar 28, 2023 •

edited

Maxim-Mazurok commented Apr 2, 2023 •

edited

nirinsanity commented Apr 26, 2023 •

edited