Pub crashes when fetching security advisories fails

[w119214972]

Environment

• Dart version (run dart --version): 2.4.0
• OS kind and version (e.g. "Windows 10, version 1809" or "macOS 12.4"): windows 10
• Are you using the Chinese community mirror or a corporate firewall?
  yes, i use "pub-web.flutter-io.cn" in my privacy package server. "pub.niimbot.info" is my privacy server which created by unpub

Problem

i upgrade my flutter to 3.22.0,
and run "flutter doctor -vv"failed.

<copy/paste pubspec.yaml>

Expected behavior

Actual behavior

HTTP response 404 Not Found for GET https://pub.niimbot.info/api/packages/archive/advisories

--trace output

IO  : Deleting directory C:\Users\liuxi\AppData\Local\Temp\pub_42df8d81.
IO  : Writing 370 characters to text file D:\MySpace\Environment\flutter_cache\README.md.
FINE: Contents:
    | Pub Package Cache
    | =================
    | 
    | This folder is used by Pub to store cached packages used in Dart / Flutter
    | projects.
    | 
    | The contents of this folder should only be modified using the `dart pub` and
    | `flutter pub` commands.
    | 
    | Modifying this folder manually can lead to inconsistent behavior.
    | 
    | For details on how manage the `PUB_CACHE`, see:
    | https://dart.dev/go/pub-cache
FINE: Downloading packages finished (2.4s).
IO  : Fetching security advisories from https://pub.niimbot.info/api/packages/archive/advisories.
IO  : HTTP GET https://pub.niimbot.info/api/packages/archive/advisories
    | Accept: application/vnd.pub.v2+json
    | user-agent: Dart pub 3.4.0
IO  : HTTP response 404 Not Found for GET https://pub.niimbot.info/api/packages/archive/advisories
    | took 0:00:00.005183
    | connection: keep-alive
    | x-powered-by: Dart with package:shelf
    | date: Wed, 15 May 2024 02:49:32 GMT
    | content-length: 15
    | content-type: text/plain; charset=utf-8
    | x-frame-options: SAMEORIGIN
    | x-xss-protection: 1; mode=block
    | x-content-type-options: nosniff
    | server: APISIX/3.3.0
ERR : HTTP error 404 Not Found
FINE: Exception type: PubHttpResponseException
ERR : package:pub/src/http.dart 412:7                        Throwing.throwIfNotOk
    | package:pub/src/http.dart 434:16                       RequestSending.fetch
    | ===== asynchronous gap ===========================
    | package:pub/src/source/hosted.dart 580:28              HostedSource._fetchAdvisories.<fn>.<fn>
    | ===== asynchronous gap ===========================
    | package:pub/src/http.dart 361:58                       retryForHttp.<fn>.<fn>
    | ===== asynchronous gap ===========================
    | package:pool/pool.dart 127:14                          Pool.withResource
    | ===== asynchronous gap ===========================
    | package:pub/src/http.dart 361:17                       retryForHttp.<fn>
    | ===== asynchronous gap ===========================
    | package:pub/src/utils.dart 749:14                      retry
    | ===== asynchronous gap ===========================
    | package:pub/src/http.dart 360:10                       retryForHttp
    | ===== asynchronous gap ===========================
    | package:pub/src/source/hosted.dart 575:16              HostedSource._fetchAdvisories.<fn>
    | ===== asynchronous gap ===========================
    | package:pub/src/authentication/client.dart 127:12      withAuthenticatedClient
    | ===== asynchronous gap ===========================
    | package:pub/src/source/hosted.dart 573:18              HostedSource._fetchAdvisories
    | ===== asynchronous gap ===========================
    | package:pub/src/source/hosted.dart 779:9               HostedSource._getAdvisories
    | ===== asynchronous gap ===========================
    | package:pub/src/source/hosted.dart 1066:24             HostedSource.getAdvisoriesForPackageVersion
    | ===== asynchronous gap ===========================
    | package:pub/src/solver/report.dart 414:26              SolveReport._reportPackage
    | ===== asynchronous gap ===========================
    | package:pub/src/solver/report.dart 218:18              SolveReport._reportChanges
    | ===== asynchronous gap ===========================
    | package:pub/src/solver/report.dart 73:21               SolveReport.show
    | ===== asynchronous gap ===========================
    | package:pub/src/entrypoint.dart 568:5                  Entrypoint.acquireDependencies
    | ===== asynchronous gap ===========================
    | package:pub/src/command/upgrade.dart 166:5             UpgradeCommand._runUpgrade
    | ===== asynchronous gap ===========================
    | package:pub/src/command/upgrade.dart 141:7             UpgradeCommand.runProtected
    | ===== asynchronous gap ===========================
    | package:pub/src/command.dart 196:7                     PubCommand.run
    | ===== asynchronous gap ===========================
    | package:args/command_runner.dart 212:13                CommandRunner.runCommand
    | ===== asynchronous gap ===========================
    | package:dartdev/dartdev.dart 232:18                    DartdevRunner.runCommand
    | ===== asynchronous gap ===========================
    | package:dartdev/dartdev.dart 46:16                     runDartdev
    | ===== asynchronous gap ===========================
    | C:\b\s\w\ir\x\w\sdk\pkg\dartdev\bin\dartdev.dart 13:5  main
ERR : This is an unexpected error. The full log and other details are collected in:
    | 
    |     D:\MySpace\Environment\flutter_cache\log\pub_log.txt
    | 
    | Consider creating an issue on https://github.com/dart-lang/pub/issues/new
    | and attaching the relevant parts of that log file.
---- End log transcript ----

[sigurdm]

@szakarias we should probably not crash when security advisories fail to load. That makes mirroring a lot harder.

[sigurdm]

To correctly mirror pub.dev you need to follow the protocol which requires any packages with a advisoriesUpdated field in the package listing to also respond to <hosted-url>/api/packages/<package>/advisories.

To resolve the problem when mirroring you should either mirror the <hosted-url>/api/packages/<package>/advisories response or strip the advisoriesUpdated field from the package listing.

Still pub should not crash here:

I think we want:

• the pub client should
  • error gracefully if the server is pub.dev
  • warn gracefully otherwise and still give a working resolution.

[szakarias]

Fixed in #4275