Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new_audit: bf-cache #14465

Merged
merged 63 commits into from
Jan 11, 2023
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
69b34b6
WIP: bfcache audit
adamraine Oct 24, 2022
648f72e
ope
adamraine Oct 24, 2022
c25b196
smoke fix
adamraine Oct 25, 2022
29352f2
ope
adamraine Oct 25, 2022
b89df49
comments
adamraine Oct 25, 2022
111abd4
tree
adamraine Oct 25, 2022
d2db291
organize results in artifact
adamraine Oct 26, 2022
7a669a0
all types
adamraine Oct 26, 2022
795593f
passive flow
adamraine Oct 27, 2022
4e4138a
Merge branch 'main' into bf-cache
adamraine Oct 28, 2022
e4aa601
semi
adamraine Nov 1, 2022
8aac07f
Merge branch 'main' into bf-cache
adamraine Nov 1, 2022
892eb53
Merge branch 'main' into bf-cache
adamraine Nov 29, 2022
14dfafe
update
adamraine Nov 29, 2022
eacc071
nit
adamraine Nov 29, 2022
c74bcad
strings
adamraine Nov 29, 2022
71bd779
smoke test
adamraine Nov 29, 2022
f1dd41a
ope
adamraine Nov 30, 2022
fc67719
ope
adamraine Nov 30, 2022
c866560
sample
adamraine Nov 30, 2022
005eabd
snap
adamraine Nov 30, 2022
b36a3d4
fix
adamraine Nov 30, 2022
209995c
ope
adamraine Nov 30, 2022
c959bd5
Merge branch 'main' into bf-cache
adamraine Dec 7, 2022
92b9c99
Merge branch 'main' into bf-cache
adamraine Dec 8, 2022
3afa9c2
artifacts
adamraine Dec 8, 2022
f320249
unit
adamraine Dec 8, 2022
cf751ef
strings
adamraine Dec 8, 2022
d74cb5e
test
adamraine Dec 8, 2022
3bc53b1
Merge branch 'main' into bf-cache
adamraine Dec 13, 2022
47f8a46
sort
adamraine Dec 13, 2022
eb2f032
bfcache
adamraine Dec 13, 2022
442c231
comments
adamraine Dec 13, 2022
b6aba19
types
adamraine Dec 13, 2022
27b096e
- and sample
adamraine Dec 13, 2022
f1122e3
dedupe
adamraine Dec 13, 2022
6a58cbd
Update core/audits/bf-cache.js
adamraine Dec 13, 2022
c245ee6
hidden thing
adamraine Dec 13, 2022
1e453a7
smoke
adamraine Dec 13, 2022
919e51f
Merge branch 'main' into bf-cache
adamraine Dec 13, 2022
be1cb7c
sample
adamraine Dec 13, 2022
a2c428a
stable check
adamraine Dec 14, 2022
fe5eacc
Merge branch 'main' into bf-cache
adamraine Dec 14, 2022
9f7c944
rm old
adamraine Dec 14, 2022
99902fd
dt e2e
adamraine Dec 14, 2022
5354cbe
test
adamraine Dec 14, 2022
922bf49
fail on any failure
adamraine Dec 14, 2022
99aa1bc
reorder
adamraine Dec 14, 2022
3e489dc
dt
adamraine Dec 15, 2022
5168bcf
Merge branch 'main' into bf-cache
adamraine Dec 15, 2022
80f5672
expectation
adamraine Dec 19, 2022
c011ba0
Merge branch 'main' into bf-cache
adamraine Jan 5, 2023
e528638
use passive
adamraine Jan 5, 2023
4caf208
tests
adamraine Jan 5, 2023
b389d38
ensure event emitted
adamraine Jan 6, 2023
21b2347
test
adamraine Jan 6, 2023
59d2aa5
bump cache
adamraine Jan 10, 2023
224e675
test
adamraine Jan 10, 2023
9f88317
simpler
adamraine Jan 10, 2023
76a16d8
e2e debug
adamraine Jan 11, 2023
8677704
comment
adamraine Jan 11, 2023
9b130b9
longer timeout
adamraine Jan 11, 2023
50fcdba
fix
adamraine Jan 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions cli/test/smokehouse/test-definitions/dobetterweb.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@
/** @type {LH.Config.Json} */
const config = {
extends: 'lighthouse:default',
settings: {
// BF cache will request the page again, initiating additional network requests.
// Disable the audit so we only detect requests from the normal page load.
skipAudits: ['bf-cache'],
},
audits: [
// Test the `ignoredPatterns` audit option.
{path: 'errors-in-console', options: {ignoredPatterns: ['An ignored error']}},
Expand Down
4 changes: 4 additions & 0 deletions cli/test/smokehouse/test-definitions/perf-budgets.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ const config = {
// webpages present here, hence the inclusion of 'best-practices'.
onlyCategories: ['performance', 'best-practices'],

// BF cache will request the page again, initiating additional network requests.
// Disable the audit so we only detect requests from the normal page load.
skipAudits: ['bf-cache'],

// A mixture of under, over, and meeting budget to exercise all paths.
budgets: [{
path: '/',
Expand Down
4 changes: 4 additions & 0 deletions cli/test/smokehouse/test-definitions/perf-fonts.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ const config = {
// webpages present here, hence the inclusion of 'best-practices'.
onlyCategories: ['performance', 'best-practices'],

// BF cache will request the page again, initiating additional network requests.
// Disable the audit so we only detect requests from the normal page load.
skipAudits: ['bf-cache'],

// A mixture of under, over, and meeting budget to exercise all paths.
budgets: [{
path: '/',
Expand Down
4 changes: 4 additions & 0 deletions cli/test/smokehouse/test-definitions/perf-frame-metrics.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ const config = {
// webpages present here, hence the inclusion of 'best-practices'.
onlyCategories: ['performance', 'best-practices'],

// BF cache will request the page again, initiating additional network requests.
// Disable the audit so we only detect requests from the normal page load.
skipAudits: ['bf-cache'],

// A mixture of under, over, and meeting budget to exercise all paths.
budgets: [{
path: '/',
Expand Down
4 changes: 4 additions & 0 deletions cli/test/smokehouse/test-definitions/perf-preload.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ const config = {
// webpages present here, hence the inclusion of 'best-practices'.
onlyCategories: ['performance', 'best-practices'],

// BF cache will request the page again, initiating additional network requests.
// Disable the audit so we only detect requests from the normal page load.
skipAudits: ['bf-cache'],

// A mixture of under, over, and meeting budget to exercise all paths.
budgets: [{
path: '/',
Expand Down
4 changes: 4 additions & 0 deletions cli/test/smokehouse/test-definitions/perf-trace-elements.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ const config = {
// webpages present here, hence the inclusion of 'best-practices'.
onlyCategories: ['performance', 'best-practices'],

// BF cache will request the page again, initiating additional network requests.
// Disable the audit so we only detect requests from the normal page load.
skipAudits: ['bf-cache'],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should have a test-definition base config we extend from? we could also specify only-categories: [] in it, forcing smoke configs that extend it to be more specific (I assume that works...)

Copy link
Member Author

@adamraine adamraine Nov 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like overkill plus it would be confusing when writing a test to bounce between the smokehouse base config and the test specific config.

This issue is only present on a handful of tests.


// A mixture of under, over, and meeting budget to exercise all paths.
budgets: [{
path: '/',
Expand Down
141 changes: 141 additions & 0 deletions core/audits/bf-cache.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
/**
* @license Copyright 2022 The Lighthouse Authors. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
*/

import {Audit} from './audit.js';
import * as i18n from '../lib/i18n/i18n.js';
import {NotRestoredReasonDescription} from '../lib/bfcache-strings.js';
adamraine marked this conversation as resolved.
Show resolved Hide resolved

/* eslint-disable max-len */
const UIStrings = {
/** TODO */
title: 'Back/forward cache is used',
/** TODO */
failureTitle: 'Back/forward cache is not used',
/** TODO */
description: 'The back/forward cache can speed up the page load after navigating away.',
adamraine marked this conversation as resolved.
Show resolved Hide resolved
/** TODO */
actionableColumn: 'Actionable failure',
/** TODO */
notActionableColumn: 'Not actionable failure',
/** TODO */
supportPendingColumn: 'Pending browser support',
/**
* @description [ICU Syntax] Label for an audit identifying the number of back/forward cache failure reasons found in the page.
*/
displayValue: `{itemCount, plural,
=1 {1 actionable failure reason}
other {# actionable failure reasons}
}`,
};
/* eslint-enable max-len */

const str_ = i18n.createIcuMessageFn(import.meta.url, UIStrings);

class BFCache extends Audit {
/**
* @return {LH.Audit.Meta}
*/
static get meta() {
return {
id: 'bf-cache',
title: str_(UIStrings.title),
failureTitle: str_(UIStrings.failureTitle),
description: str_(UIStrings.description),
supportedModes: ['navigation', 'timespan'],
requiredArtifacts: ['BFCacheErrors'],
};
}

/**
* @param {LH.Crdp.Page.BackForwardCacheNotRestoredReason} reason
*/
static getDescriptionForReason(reason) {
const matchingString = NotRestoredReasonDescription[reason];

if (matchingString === undefined) {
return reason;
}

return matchingString.name;
adamraine marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* @param {LH.Artifacts.BFCacheErrorMap} errors
* @param {LH.IcuMessage | string} label
* @return {LH.Audit.Details.Table}
*/
static makeTableForFailureType(errors, label) {
/** @type {LH.Audit.Details.TableItem[]} */
const results = [];

// https://github.com/Microsoft/TypeScript/issues/12870
const reasons = /** @type {LH.Crdp.Page.BackForwardCacheNotRestoredReason[]} */
(Object.keys(errors));

for (const reason of reasons) {
const frameUrls = errors[reason] || [];
results.push({
reason: this.getDescriptionForReason(reason),
subItems: {
type: 'subitems',
items: frameUrls.map(frameUrl => ({frameUrl})),
},
});
}

/** @type {LH.Audit.Details.Table['headings']} */
const headings = [
/* eslint-disable max-len */
{key: 'reason', valueType: 'text', subItemsHeading: {key: 'frameUrl', valueType: 'url'}, label},
/* eslint-enable max-len */
];

return Audit.makeTableDetails(headings, results);
}

/**
* @param {LH.Artifacts} artifacts
* @return {Promise<LH.Audit.Product>}
*/
static async audit(artifacts) {
const {PageSupportNeeded, SupportPending, Circumstantial} = artifacts.BFCacheErrors;

const actionableTable =
this.makeTableForFailureType(PageSupportNeeded, str_(UIStrings.actionableColumn));
const notActionableTable =
this.makeTableForFailureType(Circumstantial, str_(UIStrings.notActionableColumn));
const supportPendingTable =
this.makeTableForFailureType(SupportPending, str_(UIStrings.supportPendingColumn));

const items = [
actionableTable,
notActionableTable,
supportPendingTable,
];

if (actionableTable.items.length === 0) {
return {
score: 1,
details: {
type: 'list',
items,
},
};
}

return {
score: 0,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may want to mark n/a if supportPendingTable or notActionableTable is >0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are still actionable failures I think it's worth surfacing them

displayValue: str_(UIStrings.displayValue, {itemCount: actionableTable.items.length}),
details: {
type: 'list',
items,
adamraine marked this conversation as resolved.
Show resolved Hide resolved
},
};
}
}

export default BFCache;
export {UIStrings};
8 changes: 7 additions & 1 deletion core/config/default-config.js
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ const artifacts = {
Trace: '',
Accessibility: '',
AnchorElements: '',
BFCacheErrors: '',
CacheContents: '',
ConsoleMessages: '',
CSSUsage: '',
Expand Down Expand Up @@ -216,8 +217,11 @@ const defaultConfig = {
{id: artifacts.devtoolsLogs, gatherer: 'devtools-log-compat'},
{id: artifacts.traces, gatherer: 'trace-compat'},

// FullPageScreenshot comes at the very end so all other node analysis is captured.
// FullPageScreenshot comes at the end so all other node analysis is captured.
{id: artifacts.FullPageScreenshot, gatherer: 'full-page-screenshot'},

// BFCacheErrors comes at the very end because it can perform a page navigation.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you didn't like "very very end"? :P

{id: artifacts.BFCacheErrors, gatherer: 'bf-cache-errors'},
],
audits: [
'is-on-https',
Expand Down Expand Up @@ -373,6 +377,7 @@ const defaultConfig = {
'seo/canonical',
'seo/manual/structured-data',
'work-during-interaction',
'bf-cache',
],
groups: {
'metrics': {
Expand Down Expand Up @@ -515,6 +520,7 @@ const defaultConfig = {
{id: 'no-unload-listeners', weight: 0},
{id: 'uses-responsive-images-snapshot', weight: 0},
{id: 'work-during-interaction', weight: 0},
{id: 'bf-cache', weight: 0},

// Budget audits.
{id: 'performance-budget', weight: 0, group: 'budgets'},
Expand Down
2 changes: 1 addition & 1 deletion core/gather/driver.js
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ class Driver {
/** @return {Promise<void>} */
async disconnect() {
if (this.defaultSession === throwingSession) return;
this._targetManager?.disable();
await this._targetManager?.disable();
await this.defaultSession.dispose();
}
}
Expand Down
17 changes: 12 additions & 5 deletions core/gather/driver/target-manager.js
Original file line number Diff line number Diff line change
Expand Up @@ -58,15 +58,22 @@ class TargetManager extends ProtocolEventEmitter {
async _onFrameNavigated(frameNavigatedEvent) {
// Child frames are handled in `_onSessionAttached`.
if (frameNavigatedEvent.frame.parentId) return;
if (!this._enabled) return;

// It's not entirely clear when this is necessary, but when the page switches processes on
// navigating from about:blank to the `requestedUrl`, resetting `setAutoAttach` has been
// necessary in the past.
await this._rootCdpSession.send('Target.setAutoAttach', {
autoAttach: true,
flatten: true,
waitForDebuggerOnStart: true,
});
try {
await this._rootCdpSession.send('Target.setAutoAttach', {
autoAttach: true,
flatten: true,
waitForDebuggerOnStart: true,
});
} catch (err) {
// The page can be closed at the end of the run before this CDP function returns.
// In these cases, just ignore the error since we won't need the page anyway.
if (this._enabled) throw err;
}
}

/**
Expand Down