total-byte-weight started failing but only on the CLI

[nstringham]

FAQ

• Yes, my issue is not about variability or throttling.
• Yes, my issue is not about a specific accessibility audit (file with axe-core instead).

URL

https://othello-rust.web.app/

What happened?

recently the lhci autorun command began giving the following error

       ✘  total-byte-weight failure for minScore assertion
       Avoids enormous network payloads
       https://developer.chrome.com/docs/lighthouse/performance/total-byte-weight/

        expected: >=0.9
           found: 0
      all values: 0, 0, 0

config:

ci:
  collect:
    autodiscover-url-blocklist: /privacy-policy.html
  upload:
    target: "temporary-public-storage"
  assert:
    preset: "lighthouse:recommended"
    assertions:
      csp-xss: "off"
      bf-cache: "warn"

What did you expect?

I would expect the CLI to pass because I do not get an error about "Avoid enormous network payloads" anywhere else. including in the HTML report generated by the CLI

What have you tried?

using the lighthouse inside of chrome and using the lighthouse inside of page speed insights

How were you running Lighthouse?

CLI

Lighthouse Version

0.13.0

Chrome Version

120.0.6099.129

Node Version

v18.12.1

OS

Windows 11 23H2

Relevant log output

https://github.com/nstringham/othello-web-app/actions/runs/7315817257/job/19929865498

[davidlj95]

+1 Another job run sample:

https://github.com/davidlj95/chrislb/actions/runs/7305097678/job/19908261745?pr=264

If running locally, I can see in the total byte weight audit is around 320Kbs which shouldn't raise the audit AFAIK

Local output:

lighthouseci.zip

[nstringham]

Any update on this?

[gilhanan]

I'm also expereince this issue. If someone want to repro it easily:
npx @lhci/cli autorun --collect.url=https://gh-lhci-azure-demo-app---staging.jollyrock-437e64c9.northeurope.azurecontainerapps.io --assert.preset=lighthouse:no-pwa

  ×  total-byte-weight failure for minScore assertion
       Avoids enormous network payloads
       https://developer.chrome.com/docs/lighthouse/performance/total-byte-weight/

        expected: >=0.9
           found: 0
      all values: 0, 0, 0

[davidlj95]

After digging again into this, noticed that score is null in the JSON report

    "total-byte-weight": {
      "id": "total-byte-weight",
      "title": "Avoids enormous network payloads",
      "description": "Large network payloads cost users real money and are highly correlated with long load times. [Learn how to reduce payload sizes](https://developer.chrome.com/docs/lighthouse/performance/total-byte-weight/).",
      "score": null,
      "scoreDisplayMode": "informative",
      "numericValue": 356395,
      "numericUnit": "byte",
      "displayValue": "Total size was 348 KiB",
      "details": {
        "type": "table",
        "headings": [
          {
            "key": "url",
            "valueType": "url",
            "label": "URL"
          },
          {
            "key": "totalBytes",
            "valueType": "bytes",
            "label": "Transfer Size"
          }
        ],
        "items": [
          {
            "url": "http://localhost:4000/chunk-XALDH5GC.js",
            "totalBytes": 55713
          }
        ],
        "sortedBy": [
          "totalBytes"
        ]
      },
      "guidanceLevel": 1
    },

(removed items to paste a shorter JSON)

[davidlj95]

Seems very related to #994

[davidlj95]

Checkout for a workaround here:

#994 (comment)