Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store remote registry file hashes in the lockfile #21901

Closed
wants to merge 7 commits into from

Conversation

fmeum
Copy link
Collaborator

@fmeum fmeum commented Apr 4, 2024

MODULE.bazel, bazel_registry.json and source.json files obtained from remote registries are stored in the repository cache and their hashes are collected in the lockfile. This speeds up incremental module resolutions, such as after adding a new bazel_dep.

Yanked versions are not stored in the lockfile. Their handling will be part of a follow-up PR.

Implements part of https://docs.google.com/document/d/1TjA7-M5njkI1F38IC0pm305S9EOmxcUwaCIvaSmansg/edit
Work towards #20369

@fmeum fmeum changed the title 20369 remote file hash Store registry file hashes in the lockfile Apr 5, 2024
@fmeum fmeum changed the title Store registry file hashes in the lockfile Store remote registry file hashes in the lockfile Apr 5, 2024
@fmeum fmeum marked this pull request as ready for review April 5, 2024 16:29
@github-actions github-actions bot added team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. awaiting-review PR is awaiting review from an assigned reviewer labels Apr 5, 2024
@fmeum
Copy link
Collaborator Author

fmeum commented Apr 5, 2024

Currently stacked on #21906, but otherwise ready for review.

@fmeum
Copy link
Collaborator Author

fmeum commented Apr 17, 2024

This is no longer stacked and ready for review again.

@fmeum fmeum force-pushed the 20369-remote-file-hash branch 2 times, most recently from 9b1f024 to 4644add Compare April 20, 2024 15:36
@fmeum fmeum requested a review from a team as a code owner April 20, 2024 15:36
@fmeum fmeum requested review from gregestren and removed request for a team April 20, 2024 15:36
@fmeum fmeum removed the request for review from gregestren April 21, 2024 16:58
@fmeum fmeum marked this pull request as draft April 21, 2024 16:58
@fmeum fmeum force-pushed the 20369-remote-file-hash branch 3 times, most recently from ae53f76 to 32e04db Compare April 21, 2024 17:47
@fmeum fmeum force-pushed the 20369-remote-file-hash branch 3 times, most recently from 9767acc to 189282d Compare April 25, 2024 09:00
@fmeum fmeum force-pushed the 20369-remote-file-hash branch 2 times, most recently from 3e435a3 to 467a2a8 Compare May 2, 2024 15:24
@fmeum fmeum requested a review from Wyverald May 2, 2024 15:24
@Wyverald Wyverald added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels May 2, 2024
@iancha1992
Copy link
Member

@fmeum can you please add the Javadoc comment for KnownFileHashesMode from src/main/java/com/google/devtools/build/lib/bazel/bzlmod/IndexRegistry.java? Thanks!

@fmeum
Copy link
Collaborator Author

fmeum commented May 3, 2024

@fmeum can you please add the Javadoc comment for KnownFileHashesMode from src/main/java/com/google/devtools/build/lib/bazel/bzlmod/IndexRegistry.java? Thanks!

Done!

@Wyverald
Copy link
Member

Wyverald commented May 6, 2024

quick update -- running into some internal test failures during import; will probably submit today or tomorrow

@Wyverald
Copy link
Member

Wyverald commented May 6, 2024

For future reference:

  1. Anything contained in a SkyValue needs to implement equals() and hashCode() (Checksum did not)
  2. Our internal serialization framework doesn't support records (yet?), so we have to use AutoValue for now.

No need to make any changes -- this is just informational

@copybara-service copybara-service bot closed this in 8a81b81 May 6, 2024
@github-actions github-actions bot removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label May 6, 2024
@fmeum fmeum deleted the 20369-remote-file-hash branch May 7, 2024 07:19
@fmeum
Copy link
Collaborator Author

fmeum commented May 7, 2024

@bazel-io fork 7.2.0

Wyverald pushed a commit that referenced this pull request May 8, 2024
`MODULE.bazel`, `bazel_registry.json` and `source.json` files obtained from remote registries are stored in the repository cache and their hashes are collected in the lockfile. This speeds up incremental module resolutions, such as after adding a new `bazel_dep`.

Yanked versions are not stored in the lockfile. Their handling will be part of a follow-up PR.

Implements part of https://docs.google.com/document/d/1TjA7-M5njkI1F38IC0pm305S9EOmxcUwaCIvaSmansg/edit
Work towards #20369

Closes #21901.

PiperOrigin-RevId: 631195852
Change-Id: I35c30af7f9c3626bdbcb04c85b8c2502eeaafd3e
github-merge-queue bot pushed a commit that referenced this pull request May 8, 2024
`MODULE.bazel`, `bazel_registry.json` and `source.json` files obtained
from remote registries are stored in the repository cache and their
hashes are collected in the lockfile. This speeds up incremental module
resolutions, such as after adding a new `bazel_dep`.

Yanked versions are not stored in the lockfile. Their handling will be
part of a follow-up PR.

Implements part of
https://docs.google.com/document/d/1TjA7-M5njkI1F38IC0pm305S9EOmxcUwaCIvaSmansg/edit
Work towards #20369

Closes #21901.

PiperOrigin-RevId: 631195852
Change-Id: I35c30af7f9c3626bdbcb04c85b8c2502eeaafd3e

Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
Kila2 pushed a commit to Kila2/bazel that referenced this pull request May 13, 2024
`MODULE.bazel`, `bazel_registry.json` and `source.json` files obtained from remote registries are stored in the repository cache and their hashes are collected in the lockfile. This speeds up incremental module resolutions, such as after adding a new `bazel_dep`.

Yanked versions are not stored in the lockfile. Their handling will be part of a follow-up PR.

Implements part of https://docs.google.com/document/d/1TjA7-M5njkI1F38IC0pm305S9EOmxcUwaCIvaSmansg/edit
Work towards bazelbuild#20369

Closes bazelbuild#21901.

PiperOrigin-RevId: 631195852
Change-Id: I35c30af7f9c3626bdbcb04c85b8c2502eeaafd3e
@iancha1992
Copy link
Member

The changes in this PR have been included in Bazel 7.2.0 RC1. Please test out the release candidate and report any issues as soon as possible.
If you're using Bazelisk, you can point to the latest RC by setting USE_BAZEL_VERSION=7.2.0rc1. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants