Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tools: make utils.SearchFiles deterministic #44496

Merged
merged 1 commit into from Oct 4, 2022
Merged

tools: make utils.SearchFiles deterministic #44496

merged 1 commit into from Oct 4, 2022

Conversation

brjsp
Copy link
Contributor

@brjsp brjsp commented Sep 2, 2022

glob.glob on Linux returns files in the order returned by the filesystem driver,
and the output from this function is stuffed by the Electron build process
straight into the config.gypi header, causing non-reproducible builds.

See this log for an example of the nondeterminism:
https://rb.zq1.de/compare.factory-20220901/diffs/nodejs-electron-compare.out

@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. needs-ci PRs that need a full CI run. python PRs and issues that require attention from people who are familiar with Python. labels Sep 2, 2022
@lpinca lpinca added the request-ci Add this label to start a Jenkins CI on a PR. label Sep 2, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Sep 2, 2022
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/46393/

tniessen
tniessen approved these changes Sep 3, 2022
View reviewed changes
Copy link
Member

@tniessen tniessen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @nodejs/build

@richardlau richardlau added request-ci Add this label to start a Jenkins CI on a PR. and removed request-ci Add this label to start a Jenkins CI on a PR. labels Sep 5, 2022
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/46419/

@tniessen
Copy link
Member

tniessen commented Sep 5, 2022

@brjsp Would you mind force-pushing the same commit with an amended commit message that adheres to the commit message guidelines?

@brjsp
tools: make utils.SearchFiles deterministic
30253ac 
`glob.glob` on Linux returns files in the order returned by the
filesystem driver, and the output from this function is stuffed by
the Electron build process straight into the `config.gypi` header,
causing non-reproducible builds.

See this log for an example of the nondeterminism:
https://rb.zq1.de/compare.factory-20220901/diffs/nodejs-electron-compare.out
@brjsp
Copy link
Contributor Author

brjsp commented Sep 5, 2022

@tniessen done

@tniessen tniessen added the request-ci Add this label to start a Jenkins CI on a PR. label Sep 5, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Sep 5, 2022
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/46423/

This was referenced Sep 6, 2022
Open
Open
Open
Open
Open
Open
@brjsp brjsp requested a review from tniessen September 20, 2022 07:36
@richardlau richardlau added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 3, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 3, 2022
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/47051/

sxa
sxa approved these changes Oct 4, 2022
View reviewed changes
Copy link
Member

@sxa sxa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the function used guaranteed to be deterministic or can it be influenced by the locale setting on the machine? Either way, this will at least allow a build environment with a documented, fixed, locale to produce deterministic output, so I'm happy to approve, although there could be a small performance overhead by doing this frequently on larger directories.

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/47065/

@richardlau richardlau added the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 4, 2022
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 4, 2022
@nodejs-github-bot nodejs-github-bot merged commit e2ca29f into nodejs:main Oct 4, 2022
@nodejs-github-bot
Copy link
Collaborator

Landed in e2ca29f

@brjsp
Copy link
Contributor Author

brjsp commented Oct 4, 2022

@sxa the default string comparator in Python sorts by codepoint value, ignoring any locale settings.

danielleadams pushed a commit that referenced this pull request Oct 11, 2022
@brjsp @danielleadams
tools: make utils.SearchFiles deterministic
fd99b17 
`glob.glob` on Linux returns files in the order returned by the
filesystem driver, and the output from this function is stuffed by
the Electron build process straight into the `config.gypi` header,
causing non-reproducible builds.

See this log for an example of the nondeterminism:
https://rb.zq1.de/compare.factory-20220901/diffs/nodejs-electron-compare.out

PR-URL: #44496
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Stewart X Addison <sxa@redhat.com>
@danielleadams danielleadams mentioned this pull request Oct 11, 2022
Merged
BethGriggs pushed a commit that referenced this pull request Mar 27, 2023
@brjsp @BethGriggs
tools: make utils.SearchFiles deterministic
768e562 
`glob.glob` on Linux returns files in the order returned by the
filesystem driver, and the output from this function is stuffed by
the Electron build process straight into the `config.gypi` header,
causing non-reproducible builds.

See this log for an example of the nondeterminism:
https://rb.zq1.de/compare.factory-20220901/diffs/nodejs-electron-compare.out

PR-URL: #44496
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Stewart X Addison <sxa@redhat.com>
@BethGriggs BethGriggs mentioned this pull request Mar 27, 2023
Merged
mwalbeck pushed a commit to mwalbeck/docker-cyberchef that referenced this pull request Apr 4, 2023
@mwalbeck
Update Node.js to v16.20.0 (#187)
9ec907c 
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [node](https://github.com/nodejs/node) | stage | minor | `16.19.1-bullseye` -> `16.20.0-bullseye` |

---

### Release Notes

<details>
<summary>nodejs/node</summary>

### [`v16.20.0`](https://github.com/nodejs/node/releases/tag/v16.20.0): 2023-03-29, Version 16.20.0 &#x27;Gallium&#x27; (LTS), @&#8203;BethGriggs

[Compare Source](nodejs/node@v16.19.1...v16.20.0)

##### Notable Changes

-   **deps:**
    -   update undici to 5.20.0 (Node.js GitHub Bot) [#&#8203;46711](nodejs/node#46711)
    -   update c-ares to 1.19.0 (Michaël Zasso) [#&#8203;46415](nodejs/node#46415)
    -   upgrade npm to 8.19.4 (npm team) [#&#8203;46677](nodejs/node#46677)
    -   update corepack to 0.17.0 (Node.js GitHub Bot) [#&#8203;46842](nodejs/node#46842)
-   **(SEMVER-MINOR)** **src**: add support for externally shared js builtins (Michael Dawson) [#&#8203;44376](nodejs/node#44376)

##### Commits

-   \[[`de6dd67790`](nodejs/node@de6dd67790)] - **crypto**: avoid hang when no algorithm available (Richard Lau) [#&#8203;46237](nodejs/node#46237)
-   \[[`4617512788`](nodejs/node@4617512788)] - **crypto**: ensure auth tag set for chacha20-poly1305 (Ben Noordhuis) [#&#8203;46185](nodejs/node#46185)
-   \[[`24972164fc`](nodejs/node@24972164fc)] - **deps**: update undici to 5.20.0 (Node.js GitHub Bot) [#&#8203;46711](nodejs/node#46711)
-   \[[`85f88c6a8d`](nodejs/node@85f88c6a8d)] - **deps**: V8: cherry-pick [`90be99f`](nodejs/node@90be99fab31c) (Michaël Zasso) [#&#8203;46646](nodejs/node#46646)
-   \[[`b4ebe6d47b`](nodejs/node@b4ebe6d47b)] - **deps**: update c-ares to 1.19.0 (Michaël Zasso) [#&#8203;46415](nodejs/node#46415)
-   \[[`56cbc7fdda`](nodejs/node@56cbc7fdda)] - **deps**: V8: cherry-pick [`c2792e5`](nodejs/node@c2792e58035f) (Jiawen Geng) [#&#8203;44961](nodejs/node#44961)
-   \[[`7af9bdb31e`](nodejs/node@7af9bdb31e)] - **deps**: upgrade npm to 8.19.4 (npm team) [#&#8203;46677](nodejs/node#46677)
-   \[[`962a7471b5`](nodejs/node@962a7471b5)] - **deps**: update corepack to 0.17.0 (Node.js GitHub Bot) [#&#8203;46842](nodejs/node#46842)
-   \[[`748bc96e35`](nodejs/node@748bc96e35)] - **deps**: update corepack to 0.16.0 (Node.js GitHub Bot) [#&#8203;46710](nodejs/node#46710)
-   \[[`a467782499`](nodejs/node@a467782499)] - **deps**: update corepack to 0.15.3 (Node.js GitHub Bot) [#&#8203;46037](nodejs/node#46037)
-   \[[`1913b6763d`](nodejs/node@1913b6763d)] - **deps**: update corepack to 0.15.2 (Node.js GitHub Bot) [#&#8203;45635](nodejs/node#45635)
-   \[[`809371a15f`](nodejs/node@809371a15f)] - **module**: require.resolve.paths returns null with node schema (MURAKAMI Masahiko) [#&#8203;45147](nodejs/node#45147)
-   \[[`086bb2f8d4`](nodejs/node@086bb2f8d4)] - ***Revert*** "**src**: let http2 streams end after session close" (Rich Trott) [#&#8203;46721](nodejs/node#46721)
-   \[[`6a01d39120`](nodejs/node@6a01d39120)] - **(SEMVER-MINOR)** **src**: add support for externally shared js builtins (Michael Dawson) [#&#8203;44376](nodejs/node#44376)
-   \[[`d081032a60`](nodejs/node@d081032a60)] - **test**: fix test-net-connect-reset-until-connected (Vita Batrla) [#&#8203;46781](nodejs/node#46781)
-   \[[`efe1be47ec`](nodejs/node@efe1be47ec)] - **test**: skip test depending on `overlapped-checker` when not available (Antoine du Hamel) [#&#8203;45015](nodejs/node#45015)
-   \[[`fc47d58abe`](nodejs/node@fc47d58abe)] - **test**: remove cjs loader from stack traces (Geoffrey Booth) [#&#8203;44197](nodejs/node#44197)
-   \[[`cf76d0790d`](nodejs/node@cf76d0790d)] - **test**: fix WPT title when no META title is present (Filip Skokan) [#&#8203;46804](nodejs/node#46804)
-   \[[`0d1485b924`](nodejs/node@0d1485b924)] - **test**: fix default WPT titles (Filip Skokan) [#&#8203;46778](nodejs/node#46778)
-   \[[`088e9cde3d`](nodejs/node@088e9cde3d)] - **test**: add WPTRunner support for variants and generating WPT reports (Filip Skokan) [#&#8203;46498](nodejs/node#46498)
-   \[[`908c4dff44`](nodejs/node@908c4dff44)] - **test**: mark test-crypto-key-objects flaky on Linux (Richard Lau) [#&#8203;46684](nodejs/node#46684)
-   \[[`768e56227e`](nodejs/node@768e56227e)] - **tools**: make `utils.SearchFiles` deterministic (Bruno Pitrus) [#&#8203;44496](nodejs/node#44496)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4yMy4zIiwidXBkYXRlZEluVmVyIjoiMzUuMjMuMyJ9-->

Reviewed-on: https://git.walbeck.it/mwalbeck/docker-cyberchef/pulls/187
Co-authored-by: renovate-bot <bot@walbeck.it>
Co-committed-by: renovate-bot <bot@walbeck.it>
mwalbeck pushed a commit to mwalbeck/docker-jellyfin-livestream that referenced this pull request Apr 4, 2023
@mwalbeck
Update Node.js to v16.20.0 (#243)
31f455a 
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [node](https://github.com/nodejs/node) | stage | minor | `16.19.1-bullseye-slim` -> `16.20.0-bullseye-slim` |

---

### Release Notes

<details>
<summary>nodejs/node</summary>

### [`v16.20.0`](https://github.com/nodejs/node/releases/tag/v16.20.0): 2023-03-29, Version 16.20.0 &#x27;Gallium&#x27; (LTS), @&#8203;BethGriggs

[Compare Source](nodejs/node@v16.19.1...v16.20.0)

##### Notable Changes

-   **deps:**
    -   update undici to 5.20.0 (Node.js GitHub Bot) [#&#8203;46711](nodejs/node#46711)
    -   update c-ares to 1.19.0 (Michaël Zasso) [#&#8203;46415](nodejs/node#46415)
    -   upgrade npm to 8.19.4 (npm team) [#&#8203;46677](nodejs/node#46677)
    -   update corepack to 0.17.0 (Node.js GitHub Bot) [#&#8203;46842](nodejs/node#46842)
-   **(SEMVER-MINOR)** **src**: add support for externally shared js builtins (Michael Dawson) [#&#8203;44376](nodejs/node#44376)

##### Commits

-   \[[`de6dd67790`](nodejs/node@de6dd67790)] - **crypto**: avoid hang when no algorithm available (Richard Lau) [#&#8203;46237](nodejs/node#46237)
-   \[[`4617512788`](nodejs/node@4617512788)] - **crypto**: ensure auth tag set for chacha20-poly1305 (Ben Noordhuis) [#&#8203;46185](nodejs/node#46185)
-   \[[`24972164fc`](nodejs/node@24972164fc)] - **deps**: update undici to 5.20.0 (Node.js GitHub Bot) [#&#8203;46711](nodejs/node#46711)
-   \[[`85f88c6a8d`](nodejs/node@85f88c6a8d)] - **deps**: V8: cherry-pick [`90be99f`](nodejs/node@90be99fab31c) (Michaël Zasso) [#&#8203;46646](nodejs/node#46646)
-   \[[`b4ebe6d47b`](nodejs/node@b4ebe6d47b)] - **deps**: update c-ares to 1.19.0 (Michaël Zasso) [#&#8203;46415](nodejs/node#46415)
-   \[[`56cbc7fdda`](nodejs/node@56cbc7fdda)] - **deps**: V8: cherry-pick [`c2792e5`](nodejs/node@c2792e58035f) (Jiawen Geng) [#&#8203;44961](nodejs/node#44961)
-   \[[`7af9bdb31e`](nodejs/node@7af9bdb31e)] - **deps**: upgrade npm to 8.19.4 (npm team) [#&#8203;46677](nodejs/node#46677)
-   \[[`962a7471b5`](nodejs/node@962a7471b5)] - **deps**: update corepack to 0.17.0 (Node.js GitHub Bot) [#&#8203;46842](nodejs/node#46842)
-   \[[`748bc96e35`](nodejs/node@748bc96e35)] - **deps**: update corepack to 0.16.0 (Node.js GitHub Bot) [#&#8203;46710](nodejs/node#46710)
-   \[[`a467782499`](nodejs/node@a467782499)] - **deps**: update corepack to 0.15.3 (Node.js GitHub Bot) [#&#8203;46037](nodejs/node#46037)
-   \[[`1913b6763d`](nodejs/node@1913b6763d)] - **deps**: update corepack to 0.15.2 (Node.js GitHub Bot) [#&#8203;45635](nodejs/node#45635)
-   \[[`809371a15f`](nodejs/node@809371a15f)] - **module**: require.resolve.paths returns null with node schema (MURAKAMI Masahiko) [#&#8203;45147](nodejs/node#45147)
-   \[[`086bb2f8d4`](nodejs/node@086bb2f8d4)] - ***Revert*** "**src**: let http2 streams end after session close" (Rich Trott) [#&#8203;46721](nodejs/node#46721)
-   \[[`6a01d39120`](nodejs/node@6a01d39120)] - **(SEMVER-MINOR)** **src**: add support for externally shared js builtins (Michael Dawson) [#&#8203;44376](nodejs/node#44376)
-   \[[`d081032a60`](nodejs/node@d081032a60)] - **test**: fix test-net-connect-reset-until-connected (Vita Batrla) [#&#8203;46781](nodejs/node#46781)
-   \[[`efe1be47ec`](nodejs/node@efe1be47ec)] - **test**: skip test depending on `overlapped-checker` when not available (Antoine du Hamel) [#&#8203;45015](nodejs/node#45015)
-   \[[`fc47d58abe`](nodejs/node@fc47d58abe)] - **test**: remove cjs loader from stack traces (Geoffrey Booth) [#&#8203;44197](nodejs/node#44197)
-   \[[`cf76d0790d`](nodejs/node@cf76d0790d)] - **test**: fix WPT title when no META title is present (Filip Skokan) [#&#8203;46804](nodejs/node#46804)
-   \[[`0d1485b924`](nodejs/node@0d1485b924)] - **test**: fix default WPT titles (Filip Skokan) [#&#8203;46778](nodejs/node#46778)
-   \[[`088e9cde3d`](nodejs/node@088e9cde3d)] - **test**: add WPTRunner support for variants and generating WPT reports (Filip Skokan) [#&#8203;46498](nodejs/node#46498)
-   \[[`908c4dff44`](nodejs/node@908c4dff44)] - **test**: mark test-crypto-key-objects flaky on Linux (Richard Lau) [#&#8203;46684](nodejs/node#46684)
-   \[[`768e56227e`](nodejs/node@768e56227e)] - **tools**: make `utils.SearchFiles` deterministic (Bruno Pitrus) [#&#8203;44496](nodejs/node#44496)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4yMy4zIiwidXBkYXRlZEluVmVyIjoiMzUuMjMuMyJ9-->

Reviewed-on: https://git.walbeck.it/mwalbeck/docker-jellyfin-livestream/pulls/243
Co-authored-by: renovate-bot <bot@walbeck.it>
Co-committed-by: renovate-bot <bot@walbeck.it>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lpinca lpinca lpinca approved these changes

@richardlau richardlau richardlau approved these changes

@sxa sxa sxa approved these changes

@tniessen tniessen Awaiting requested review from tniessen

Assignees
No one assigned
Labels
build Issues and PRs related to build files or the CI. needs-ci PRs that need a full CI run. python PRs and issues that require attention from people who are familiar with Python.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@brjsp @nodejs-github-bot @tniessen @lpinca @richardlau @sxa