You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Due to variability, Lighthouse CI may sometimes report a lower performance score that causes an assertion to fail, whereas for most runs, the assertion passes. Since the Lighthouse test takes rather long, I've configured numberOfRuns: 1 for my CI pipeline. However, if, and ideally only if, a performance assertion fails, it would be helpful if the test could be re-run for a given URL.
Increasing numberOfRuns isn't quite the same thing, as it will run multiple times also when the first test passes.
Describe the solution you'd like
An option to retry the assertions for a given page if the the first run fails.
Describe alternatives you've considered
Wrapping the entire LHCI run to re-run if it fails (this needlessly tests URLs again that have already passed).
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Due to variability, Lighthouse CI may sometimes report a lower performance score that causes an assertion to fail, whereas for most runs, the assertion passes. Since the Lighthouse test takes rather long, I've configured
numberOfRuns: 1
for my CI pipeline. However, if, and ideally only if, a performance assertion fails, it would be helpful if the test could be re-run for a given URL.Increasing
numberOfRuns
isn't quite the same thing, as it will run multiple times also when the first test passes.Describe the solution you'd like
An option to retry the assertions for a given page if the the first run fails.
Describe alternatives you've considered
Wrapping the entire LHCI run to re-run if it fails (this needlessly tests URLs again that have already passed).
The text was updated successfully, but these errors were encountered: