Handle pending connections losing context on frame navigation

[francisbeaudoin]

Details

Similar to what is being proposed in #420, this PR fixes intermittent Ferrum::TimeoutError as the driver awaits for requests to be completed.

What

It was observed that in some scenarios the driver is not always receiving and associated Network.loadingFailed for requests that were in-flight as the main frame navigates. This may be happening with either main frame requests or sub-frame ones.

I'm not sure if this is the right approach, but it appears to be working. Basically, it flags requests which are no longer relevant to the current page context as unknown which are then assumed to be #finished.

Relates to #420

[route]

@francisbeaudoin do you have an example of such a page in the public internet? I just want to check the CDP logs to better understand of what's going on. A test describing the issue would be more appropriate though.

[francisbeaudoin]

@route Thanks for following up.

I spent some time reproducing the issue by using a use case that was causing timeouts at the time, and found something interesting.

By using the headless browser instance opened by ferrum vs using a regular instance ran from the desktop, the network requests were not returning the same status. i.e. (canceled) vs (unknown). As far as I remember, the (unknown) ones were causing the library to raise a timeout as it couldn't infer if a request had completed.

I've identified the different behavior to the disable-site-isolation-trials option. If that option is removed thus enabled, Chrome shows a (unknown) network request as opposed to (canceled) status. In the current library state, the option is disabled (added 4 months ago) so it may not longer be an issue for this specific use case, but I suspect there are other scenarios where requests may end up in an unknown state thereby timing out.

This was reproduced by visiting this example, and toggling the disable-site-isolation-trials option.

With disable-site-isolation-trials

Without disable-site-isolation-trials

[route]

Hmmmm, nice thank you! Let me experiment a bit with that

[francisbeaudoin]

👋 @route By any chance you had time to look into this? I'd love to stop using a forked version in my project.

[route]

Sorry not yet, was busy doing other things. Thanks for the reminder I will