New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bootstrap: make snapshot reproducible #50983
Open
joyeecheung
wants to merge
4
commits into
nodejs:main
Choose a base branch
from
joyeecheung:context-slot
base: main
Could not load branches
Branch not found: {{ refName }}
Could not load tags
Nothing to show
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Review requested:
|
nodejs-github-bot
added
lib / src
Issues and PRs related to general changes in the lib or src directory.
needs-ci
PRs that need a full CI run.
labels
Nov 30, 2023
This was referenced Nov 30, 2023
great work! keep it up! |
joyeecheung
force-pushed
the
context-slot
branch
from
March 22, 2024 19:05
7559a8a
to
6df2ede
Compare
Original commit message: [snapshot] support context embedder data serialization Previously all context embedder data are serialized verbatim, so if a context embedder data slot contains a pointer, embedders need to figure out how to store data for it on the side for the snapshot. They also have to reset the pointers to null to make the snapshot reproducible. To address this issue this patch adds v8::(Des|S)erializeContextDataCallback as part of v8::(Des|S)erializeEmbedderFieldsCallback and use them during (de)serialization of context embedder data. The bahavior is similar to that of v8::(Des|S)erializeInternalFieldsCallback: If the returned data is not empty, the returned data is going to be written into the snapshot blob and passed to the embedder during deserialization. Otherwise the data is serialized verbatim. Change-Id: I4113c76fc6dc8e3503cde5533320402454e62097 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5074252 Reviewed-by: Igor Sheludko <ishell@chromium.org> Reviewed-by: Leszek Swirski <leszeks@chromium.org> Commit-Queue: Joyee Cheung <joyee@igalia.com> Cr-Commit-Position: refs/heads/main@{#92675} Refs: v8/v8@b639938
Original commit message: [snapshot] allow NativeContext serialization to be deferred To serialize the context embedder data, we just need the back reference to the embedder data array. The context can be deferred. Bug: v8:14662 Change-Id: I6a3c9f35596cd17b7e6bc773cf598922b5b7488d Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5367362 Reviewed-by: Igor Sheludko <ishell@chromium.org> Commit-Queue: Joyee Cheung <joyee@igalia.com> Reviewed-by: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/main@{#92892} Refs: v8/v8@7fc698c
joyeecheung
force-pushed
the
context-slot
branch
from
March 22, 2024 19:07
6df2ede
to
8dc86b6
Compare
joyeecheung
changed the title
WIP: reproducible snapshot
bootstrap: make snapshot reproducible
Mar 22, 2024
joyeecheung
added
request-ci
Add this label to start a Jenkins CI on a PR.
and removed
request-ci
Add this label to start a Jenkins CI on a PR.
labels
Mar 22, 2024
joyeecheung
force-pushed
the
context-slot
branch
from
March 22, 2024 19:10
8dc86b6
to
c92d201
Compare
- Print offsets in blob serializer - Add a special node:generate_default_snapshot ID to generate the built-in snapshot. - Improve logging
joyeecheung
force-pushed
the
context-slot
branch
from
March 22, 2024 19:11
c92d201
to
2d911be
Compare
github-actions
bot
removed
the
request-ci
Add this label to start a Jenkins CI on a PR.
label
Mar 22, 2024
joyeecheung
force-pushed
the
context-slot
branch
from
March 22, 2024 22:13
2d911be
to
e0b6197
Compare
Somehow this is reproducible locally (macOS + x64 & arm64), and in the CI, on FreeBSD and SmartOS, but not on others. There are still 3-4 words of differences in the snapshot on those platforms. I'll investigate. |
This patch uses the new V8 API to {de}serialize context slots for snapshot in order to make the snapshot reproducible. Also added a test for the reproducibility of snapshots.
joyeecheung
force-pushed
the
context-slot
branch
from
March 27, 2024 14:54
e0b6197
to
9c8ab5e
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
deps: V8: cherry-pick b639938e99fa
Original commit message:
Refs: v8/v8@b639938
deps: V8: cherry-pick 7fc698c9039d
Original commit message:
Refs: v8/v8@7fc698c
src: add utilities to help debugging reproducibility of snapshots
the built-in snapshot.
bootstrap: make snapshot reproducible
This patch uses the new V8 API to {de}serialize context slots for
snapshot in order to make the snapshot reproducible. Also
added a test for the reproducibility of snapshots.
Refs: nodejs/build#3043