PWA that polls the server never updates to a newer version

[tmtron]

🐞 bug report

Affected Package

@angular/service-worker

Is this a regression?

Not sure.

Description

When the current app version constantly polls the server and this request always fails, the PWA will never update to the newer version.

Details

We have an active version of our app which starts polling the server right away to get some basic info - when the request fails, it retries after a second until the request succeeds.
We have had multiple versions of our app with this polling in place and the updates always worked fine.
Not sure if it's relevant, but our app uses registrationStrategy: 'registerImmediately'

In the newest version, we have removed this polling feature: i.e. the old endpoint does not exist anymore on the server.
The problem now is, that the PWA never updates to the newer version: i.e. the browser still shows the old version which tries to poll the (non-existent) endpoint, and since this fails the app will retry after 1 second and so on.

More info:

• The update only fails when the polling request constantly fails.
• When the app does not poll (or when it polls and the request returns okay), the update works as expected.
• Reloading the page does not update to the newer version (pressing F5)
• Even closing the browser tabs or the whole browser (close of quit) does not work.
• Hard-reload does update to the newer version (pressing CTRL+F5)

🔬 Minimal Reproduction

Here is a minimal project to reproduce the issue: https://github.com/tmtron/pwa-test
The readme contains detailed information about how to build and reproduce the update issue.

🔥 Exception or Error

PWA does not update to new version

🌍 Your Environment

Angular Version:

$ ng version

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/
    

Angular CLI: 11.0.4
Node: 14.14.0
OS: win32 x64

Angular: 11.0.4
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router
Ivy Workspace: 

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1100.4
@angular-devkit/build-angular   0.1100.4
@angular-devkit/core            11.0.4
@angular-devkit/schematics      11.0.4
@angular/service-worker         11.0.5
@schematics/angular             11.0.4
@schematics/update              0.1100.4
rxjs                            6.5.5
typescript                      4.0.5

This also happens in our production app which uses angular 10.

Anything else relevant?
Note, that the reproduction project uses the http-server package. We used this because it makes it easy to reproduce the issue without the need for a full web-server. In production we use ngingx.

Related Stackoverflow question: PWA that polls the server never updates to a newer version

[gkalpak]

Thx for the very detailed reproduction, @tmtron ❤️ 🤩 💯
It really helped identify the issue.

So, here is what happens:
In order to avoid interfering with the application requests, the SW will not immediately check for updates on initialization and on navigation requests. Insted it will schedule a task to check for updates when "idle". Currently, the SW is considered idle when it hasn't received any request for 5 seconds.

So, since your app is constantly polling (i.e. sending requests), the SW will never consider itself idle and therefore will never perform the check-for-updates task.

Note: This is not related to the requests failing or succeeding. It just happens that when the request fails, your app will send a new one before the 5s idle threshold.

Despite the fact that it is probably not a good idea for an app to constantly make the same request when it fails (for example, there should be some backoff mechanism), the SW should be able to handle that more gracefully 😁

I will send a PR to make it so that the IdleScheduler has a max delay (say 30s) after which it executes its pending tasks even if the SW is not considered "idle".

As far as work-arounds go, there are several things one could do on the client to avoid this issue:

1. Call SwUpdate#checkForUpdate() to trigger an immediate check for updates. This does not go through the IdleScheduler, so it is not subject to the issue.

2. Use ngsw-bypass on the offending request to bypass the SW (which means that the recurring request would not prevent the SW from being considered idle).

3. Avoid polling at a constant rate when the request fails and implement some sort of exponential backoff.

Of course, none of these will help you if you can't update your client code (because the SW keeps serving the old version) 😅

I am afraid, the only thing you can do from the server is return a 2xx response for the polling request (so that the app stops polling and the SW can update itself.

Alternatively (or in addition) you could configure the server to include the Clear-Site-Data header (with the "storage" or "*" directives) for the removed endpoint, so that old clients who make that request will have their SW unregistered by the browser.

[tmtron]

Thanks for the fast and detailed response.
Now I understand what's going on.

Despite the fact that it is probably not a good idea for an app to constantly make the same request when it fails (for example, there should be some backoff mechanism)

Funny, that exactly this "improvement" has now caused this issue :)
In the newer versions of the app the polling was replaced by a websocket connection. Fortunately the service worker does not interfere with the websocket connection (which also tries to reconnect constantly)

the SW should be able to handle that more gracefully
I will send a PR to make it so that the IdleScheduler has a max delay (say 30s) after which it executes its pending tasks even if the SW is not considered "idle".

Sounds like a good idea, but as always, defining the right timeout is tricky. e.g.

• when a user (re-)visits a page that does not work, they may refresh the page (which does not help in this case). This user-interaction may only take some seconds (3?, 5?)
• or they may try to close and reopen the page/browser (which does also not help)
• even when the users retry the next day, they still have the same procedure: check, refresh, leave (in let's say 5 seconds).
• I don't think that there are many users that keep a "broken" page open for 30 seconds or more.

1. Call SwUpdate#checkForUpdate() to trigger an immediate check for updates. This does not go through the IdleScheduler, so it is not subject to the issue.

That's what our (old) app does, but unfortunately, it waits until the angular app is idle, as mentioned in the docs: https://angular.io/guide/service-worker-communications#checking-for-updates
And since our timer constantly restarts, the app never becomes idle :(

So in the new version, we will check for updates right away, no matter if the app is stable or not.
When there is a new version we must anyway update ASAP, because requests from the old client might introduce issues when they are sent to a newer server version...

I am afraid, the only thing you can do from the server is return a 2xx response for the polling request (so that the app stops polling and the SW can update itself.

Yes, that's also the workaround that we came up with. And it essentially means that we can never remove this endpoint from our server app.

So from my side, the issue is resolved. Thanks!

[gkalpak]

Thx for getting back, @tmtron!

Sounds like a good idea, but as always, defining the right timeout is tricky. e.g.

• when a user (re-)visits a page that does not work, they may refresh the page (which does not help in this case). This user-interaction may only take some seconds (3?, 5?)
• or they may try to close and reopen the page/browser (which does also not help)
• even when the users retry the next day, they still have the same procedure: check, refresh, leave (in let's say 5 seconds).
• I don't think that there are many users that keep a "broken" page open for 30 seconds or more.

These would be valid concerns, but fortunately this is not how SWs work 😃
The SW instance will remain alive after the page is closed. Especially since we call event.waitUntil(this.idle.trigger()) (for example, here and elsewhere), the SW instance will remain alive until the promise is resolved (i.e. until the scheduled tasks have ben completed).

So, if we add a 30s max delay, at most 30s after the user opens the broken page, the tasks will run (regardless of what the user does, i.e. regardless of whether the user stays on the page or not).

Note that the max delay is only relevant if the user stays on a page that continues to make requests (which prevents the SW from going idle). If the user closes the page (i.e. the SW stops receiving requests), then the scheduled tasks will be executed after 5 secs.

That's what our (old) app does, but unfortunately, it waits until the angular app is idle

Obviously, that technique will only work if the app does become stable at some point 😁
Feel free to send a pull request updating the docs if you think there is room for clarification there.

I am going to close this issue since you have been able to solve it on your end and there is nothing that can be done to fix the issue from the client-side at this point. (Feel free to continue the discussion below, if needed.)

I'll send the PR to add the max delay in IdleScheduler in order to avoid similar issues in the future.

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}