Stablize and flip repository_cache_urls_as_default_canonical_id

[sluongng]

Introduced in #14268, this flag is very useful for bigger enterprise
context where folks often version bumping dependencies without
remembering to update the SHA256 of the downloaded file, leading to
Bazel picking up older download entries from the repository cache.

As we get more questions about this flag in Slack, marking it as
stable and flip the default seems to be the right move.

[sluongng]

Not quite sure what's up with Bazel CI, a lot of failures with Unknown host: github.com and I don't have permission to retry on buildkite

[sluongng]

@meteorcloudy pinging you for review since you approved the last PR.

I think all the repository downloads failures on CI are the ones that are fetching directly from github and do not have a mirror.

[meteorcloudy]

We run most of our integration tests without internet access on BazelCI, to do that, we passed the repo cache generated by the outter Bazel (6.3.2) to the inner Bazel (HEAD) inside the integration test.

I guess this change caused the repo cache not to work when the URL differs (mirror.bazel.build vs github.com) for the same artifact.

The simplest work around would be to turn off this flag for integration tests at all places we set the repo cache:
https://cs.opensource.google/search?q=%22common%20--repository_cache%22&ss=bazel
Otherwise, we'll have to figure out a way to make sure the outter Bazel also fetches artifact with this flag on and at the same URL.

[meteorcloudy]

Probably also here:

bazel/scripts/bootstrap/bootstrap.sh

Line 38 in 62386d9

--repository_cache=derived/repository_cache \

[sluongng]

@meteorcloudy CI passed for now. Do let me know if you prefer to move the change from bazel_bootstrap_distfile_test.sh to bazel/scripts/bootstrap/bootstrap.sh! 🙇

[meteorcloudy]

Yes, please. Because if users just run the bootstrap build following https://bazel.build/install/compile-source#bootstrap-unix, they won't have the flag set, so their build will likely to fail without internet access. Let's not break that use case.

[thesayyn]

I believe this is a mistake as it will lead to more disk space usage and performance degradation as identical blobs are downloaded even though they exist in the cache.

See: #19749

[fmeum]

I believe this is a mistake as it will lead to more disk space usage and performance degradation as identical blobs are downloaded even though they exist in the cache.

I supported the flip as it would have caught mistakes I made at numerous occasions, but I can see how it can also be problematic. I would propose to revert the flip and stabilization and rethink the flag -are canonical IDs really the right approach to catch the common mistake of forgetting to update a hash after updating a URL? I could see possible answers being that we want to limit the reach of this flag to http_* rules only or come with a way to show a warning that doesn't result in cache entry duplication.