Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(cli): refactor js-binding to support easier bundling. #1957

Merged
merged 4 commits into from
Feb 26, 2024

Conversation

everett1992
Copy link
Contributor

#1948

This PR addresses two issues I had with using the JavaScript generated by @napi-rs/cli with a bundler.

  1. The existsSync check before require breaks if the bundler renames the .node file
  2. platform, arch cannot be defined at build time.

With this change esbuild can bundle napi-rs libraries with just --loader:.node=copy and webpack can bundle with node-loader.
Before I was able to use esbuild by disabling asset hashing, but I wasn't able to get webpack working.

Additionally you can define process.arch and process.platform at build time and requires to unused platforms/arch's will be eliminated and not included in the bundle.

@Brooooooklyn
Copy link
Sponsor Member

NAPI-RS had a similar implementation before; but it breaks another tools like ncc on Vercel: napi-rs/node-rs#316

@everett1992
Copy link
Contributor Author

Do you have a link to that similar implementation?

I tested this PR with @vercel/ncc, esbuild, and webpack with both grouped and separated package binaries.
This implementation seems more likely to work across all bundlers because it doesn't use any require.resolve or file system checks. It's the same pattern you're currently using for separated package binaries, but I use it for both separated and grouped, and skip the file exists check.

 function requireNative() {
  if (process.platform === 'android') {
    if (process.arch === 'arm64') {
      try {
        return require('./rust-cloudauth-core.android-arm64.node')
      } catch (e) {
        loadErrors.push(e)
      }
      try {
        return require('@amzn/rust-cloudauth-core-android-arm64')
      } catch (e) {
        loadErrors.push(e)
      }

@Brooooooklyn
Copy link
Sponsor Member

Do you have a link to that similar implementation?

https://github.com/napi-rs/node-rs/blob/main/packages/helper/src/loader.ts

For loading logic like this:

function requireNative() {
  if (process.platform === 'android') {
    if (process.arch === 'arm64') {
      try {
        return require('./rust-cloudauth-core.android-arm64.node')
      } catch (e) {
        loadErrors.push(e)
      }
      try {
        return require('@amzn/rust-cloudauth-core-android-arm64')
      } catch (e) {
        loadErrors.push(e)
      }

I'm worried that such logic is too dynamic, some packing tools that are not implemented finely may not be able to analyze the actual path/package required.

@everett1992
Copy link
Contributor Author

everett1992 commented Feb 23, 2024 via email

The existsSync check breaks the bundlers I've tested (esbuild, webpack,
ncc), so you cannot bundle napi-rs packages that have all binarys in a
single package.

I've tested this change with both single package and multi package
libraries.
@Brooooooklyn Brooooooklyn changed the title Refactor js-binding to support easier bundling. refactor(cli): refactor js-binding to support easier bundling. Feb 26, 2024
@Brooooooklyn Brooooooklyn merged commit 95dd6ef into napi-rs:main Feb 26, 2024
41 of 42 checks passed
Brooooooklyn added a commit to toeverything/AFFiNE that referenced this pull request Feb 29, 2024
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [@aws-sdk/client-s3](https://togithub.com/aws/aws-sdk-js-v3/tree/main/clients/client-s3) ([source](https://togithub.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-s3)) | [`3.515.0` -> `3.523.0`](https://renovatebot.com/diffs/npm/@aws-sdk%2fclient-s3/3.515.0/3.523.0) | [![age](https://developer.mend.io/api/mc/badges/age/npm/@aws-sdk%2fclient-s3/3.523.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/npm/@aws-sdk%2fclient-s3/3.523.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/npm/@aws-sdk%2fclient-s3/3.515.0/3.523.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/@aws-sdk%2fclient-s3/3.515.0/3.523.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) |
| [@napi-rs/cli](https://togithub.com/napi-rs/napi-rs) | [`3.0.0-alpha.41` -> `3.0.0-alpha.43`](https://renovatebot.com/diffs/npm/@napi-rs%2fcli/3.0.0-alpha.41/3.0.0-alpha.43) | [![age](https://developer.mend.io/api/mc/badges/age/npm/@napi-rs%2fcli/3.0.0-alpha.43?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/npm/@napi-rs%2fcli/3.0.0-alpha.43?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/npm/@napi-rs%2fcli/3.0.0-alpha.41/3.0.0-alpha.43?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/@napi-rs%2fcli/3.0.0-alpha.41/3.0.0-alpha.43?slim=true)](https://docs.renovatebot.com/merge-confidence/) |
| [@nx/vite](https://nx.dev) ([source](https://togithub.com/nrwl/nx/tree/HEAD/packages/vite)) | [`18.0.4` -> `18.0.5`](https://renovatebot.com/diffs/npm/@nx%2fvite/18.0.4/18.0.5) | [![age](https://developer.mend.io/api/mc/badges/age/npm/@nx%2fvite/18.0.5?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/npm/@nx%2fvite/18.0.5?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/npm/@nx%2fvite/18.0.4/18.0.5?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/@nx%2fvite/18.0.4/18.0.5?slim=true)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>aws/aws-sdk-js-v3 (@&#8203;aws-sdk/client-s3)</summary>

### [`v3.523.0`](https://togithub.com/aws/aws-sdk-js-v3/blob/HEAD/clients/client-s3/CHANGELOG.md#35230-2024-02-27)

[Compare Source](https://togithub.com/aws/aws-sdk-js-v3/compare/v3.521.0...v3.523.0)

**Note:** Version bump only for package [@&#8203;aws-sdk/client-s3](https://togithub.com/aws-sdk/client-s3)

### [`v3.521.0`](https://togithub.com/aws/aws-sdk-js-v3/blob/HEAD/clients/client-s3/CHANGELOG.md#35210-2024-02-23)

[Compare Source](https://togithub.com/aws/aws-sdk-js-v3/compare/v3.515.0...v3.521.0)

##### Features

-   requestHandler ctor param pass-through ([#&#8203;5820](https://togithub.com/aws/aws-sdk-js-v3/issues/5820)) ([9fec71d](https://togithub.com/aws/aws-sdk-js-v3/commit/9fec71d1933cd8e3db118c164bca16edc2305532))

</details>

<details>
<summary>napi-rs/napi-rs (@&#8203;napi-rs/cli)</summary>

### [`v3.0.0-alpha.43`](https://togithub.com/napi-rs/napi-rs/releases/tag/%40napi-rs/cli%403.0.0-alpha.43)

[Compare Source](https://togithub.com/napi-rs/napi-rs/compare/@napi-rs/cli@3.0.0-alpha.42...@napi-rs/cli@3.0.0-alpha.43)

##### What's Changed

-   fix(cli): cleanup js binding template by [@&#8203;Brooooooklyn](https://togithub.com/Brooooooklyn) in [napi-rs/napi-rs#1984

**Full Changelog**: https://github.com/napi-rs/napi-rs/compare/[@&#8203;napi-rs/cli](https://togithub.com/napi-rs/cli)[@&#8203;3](https://togithub.com/3).0.0-alpha.42...[@&#8203;napi-rs/cli](https://togithub.com/napi-rs/cli)[@&#8203;3](https://togithub.com/3).0.0-alpha.43

### [`v3.0.0-alpha.42`](https://togithub.com/napi-rs/napi-rs/releases/tag/%40napi-rs/cli%403.0.0-alpha.42)

[Compare Source](https://togithub.com/napi-rs/napi-rs/compare/@napi-rs/cli@3.0.0-alpha.41...@napi-rs/cli@3.0.0-alpha.42)

**Full Changelog**: https://github.com/napi-rs/napi-rs/compare/napi-derive@2.16.0...[@&#8203;napi-rs/cli](https://togithub.com/napi-rs/cli)[@&#8203;3](https://togithub.com/3).0.0-alpha.42

##### What's Changed

-   chore: fix renovate path and dedupe electron versions by [@&#8203;Brooooooklyn](https://togithub.com/Brooooooklyn) in [napi-rs/napi-rs#1974
-   refactor(cli): refactor js-binding to support easier bundling. by [@&#8203;everett1992](https://togithub.com/everett1992) in [napi-rs/napi-rs#1957

##### New Contributors

-   [@&#8203;everett1992](https://togithub.com/everett1992) made their first contribution in [napi-rs/napi-rs#1957

**Full Changelog**: https://github.com/napi-rs/napi-rs/compare/napi@2.15.4...[@&#8203;napi-rs/cli](https://togithub.com/napi-rs/cli)[@&#8203;3](https://togithub.com/3).0.0-alpha.42

</details>

<details>
<summary>nrwl/nx (@&#8203;nx/vite)</summary>

### [`v18.0.5`](https://togithub.com/nrwl/nx/releases/tag/18.0.5)

[Compare Source](https://togithub.com/nrwl/nx/compare/18.0.4...18.0.5)

#### 18.0.5 (2024-02-24)

##### 🩹 Fixes

-   **angular:** do not add target defaults for the ng-packagr-lite executor when generating non-buildable library ([#&#8203;21935](https://togithub.com/nrwl/nx/pull/21935))
-   **angular:** ensure generated editor tsconfig in apps only include runtime files ([#&#8203;21945](https://togithub.com/nrwl/nx/pull/21945))
-   **core:** run migrations ordered by their target version ([#&#8203;21799](https://togithub.com/nrwl/nx/pull/21799))
-   **core:** Update NxWelcome connect to cloud ([#&#8203;21830](https://togithub.com/nrwl/nx/pull/21830))
-   **core:** propagate `verbose` flag when running `init` generator dur… ([#&#8203;21868](https://togithub.com/nrwl/nx/pull/21868))
-   **core:** ensure migrate works with yarn PnP ([#&#8203;21824](https://togithub.com/nrwl/nx/pull/21824))
-   **core:** align terminal output padding and remove leading arrow ([#&#8203;21809](https://togithub.com/nrwl/nx/pull/21809))
-   **core:** read all targets from package json when defining target defaults ([#&#8203;21719](https://togithub.com/nrwl/nx/pull/21719))
-   **core:** include nx/nuxt in migrations ([#&#8203;21885](https://togithub.com/nrwl/nx/pull/21885))
-   **core:** do not use the new pty function for older versions of windows ([#&#8203;21854](https://togithub.com/nrwl/nx/pull/21854))
-   **core:** normalize migration target versions when sorting migrations ([#&#8203;21967](https://togithub.com/nrwl/nx/pull/21967))
-   **core:** target defaults application shouldn't include extra scripts ([#&#8203;21970](https://togithub.com/nrwl/nx/pull/21970))
-   **core:** update generated README pages with more useful instructions ([#&#8203;21976](https://togithub.com/nrwl/nx/pull/21976))
-   **devkit:** respect expectComments when parsing json ([#&#8203;21584](https://togithub.com/nrwl/nx/pull/21584))
-   **graph:** fix open project with / in name ([#&#8203;21722](https://togithub.com/nrwl/nx/pull/21722))
-   **js:** nx release-version resolve-version-spec should normalize fetchSpec ([#&#8203;21710](https://togithub.com/nrwl/nx/pull/21710))
-   **js:** swc executor should support inlining on windows ([#&#8203;21801](https://togithub.com/nrwl/nx/pull/21801))
-   **js:** set moduleResolution to Node10 so it is compatible with CommonJS module ([2dac233cf](https://togithub.com/nrwl/nx/commit/2dac233cf))
-   **linter:** fix eslint-plugin migration target version ([#&#8203;21966](https://togithub.com/nrwl/nx/pull/21966))
-   **misc:** logs from rm-default-collection should render properly ([#&#8203;21953](https://togithub.com/nrwl/nx/pull/21953))
-   **misc:** set nx property in root package.json when no replacing script in nx init ([#&#8203;21974](https://togithub.com/nrwl/nx/pull/21974))
-   **nextjs:** Svg should work when svgr is true in next config ([#&#8203;21761](https://togithub.com/nrwl/nx/pull/21761))
-   **nextjs:** Add missing e2e-ci target for cypress ([#&#8203;21805](https://togithub.com/nrwl/nx/pull/21805))
-   **nuxt:** init generator should add [@&#8203;nx/vite](https://togithub.com/nx/vite) to dependencies ([#&#8203;21911](https://togithub.com/nrwl/nx/pull/21911))
-   **nuxt:** turn on autoimport ([#&#8203;21894](https://togithub.com/nrwl/nx/pull/21894))
-   **nuxt:** tsconfig types and output dir ([#&#8203;21934](https://togithub.com/nrwl/nx/pull/21934))
-   **react:** generate correctly when --js is used for module federation host/remote ([#&#8203;20119](https://togithub.com/nrwl/nx/pull/20119))
-   **react:** full support custom secure host for module federation ([#&#8203;21777](https://togithub.com/nrwl/nx/pull/21777))
-   **react:** ensure playwright configuration is using correct port in app gen ([#&#8203;21941](https://togithub.com/nrwl/nx/pull/21941))
-   **react-native:** change gradlew to absolute path ([#&#8203;21725](https://togithub.com/nrwl/nx/pull/21725))
-   **react-native:** add all flag to sync-deps ([#&#8203;21821](https://togithub.com/nrwl/nx/pull/21821))
-   **release:** skip prompt for publish when no version created ([#&#8203;21769](https://togithub.com/nrwl/nx/pull/21769))
-   **release:** use --first-parent to support merged repos ([#&#8203;21686](https://togithub.com/nrwl/nx/pull/21686))
-   **release:** move github release creation to git tasks ([#&#8203;21510](https://togithub.com/nrwl/nx/pull/21510))
-   **remix:** should add remix plugin to nx.json on init correctly ([#&#8203;21827](https://togithub.com/nrwl/nx/pull/21827))
-   **remix:** the output path should respect the remix.config.js in crystal ([#&#8203;21842](https://togithub.com/nrwl/nx/pull/21842))
-   **remix:** adjust remix start script when building ([#&#8203;21883](https://togithub.com/nrwl/nx/pull/21883))
-   **remix:** typo in tsconfig.spec.json update led to invalid tsconfig ([#&#8203;21886](https://togithub.com/nrwl/nx/pull/21886))
-   **repo:** update browser tools to fix ci ([#&#8203;21955](https://togithub.com/nrwl/nx/pull/21955))
-   **testing:** jest should handle root jest.preset.cjs ([#&#8203;21746](https://togithub.com/nrwl/nx/pull/21746))
-   **testing:** fix cypress project targets does not exist ([#&#8203;21785](https://togithub.com/nrwl/nx/pull/21785))
-   **testing:** pin cypress version to avoid issue with verifying cypress ([#&#8203;21917](https://togithub.com/nrwl/nx/pull/21917))
-   **testing:** ensure baseUrl is not passed to playwright cli ([#&#8203;21943](https://togithub.com/nrwl/nx/pull/21943))
-   **testing:** playwright plugin enoent error ([#&#8203;21951](https://togithub.com/nrwl/nx/pull/21951))
-   **testing:** add null checks when reading targets ([#&#8203;21952](https://togithub.com/nrwl/nx/pull/21952))
-   **vite:** normalize vitest cli args in executor ([#&#8203;21870](https://togithub.com/nrwl/nx/pull/21870))
-   **vite:** project conversion generator ([#&#8203;21646](https://togithub.com/nrwl/nx/pull/21646))
-   **vite:** update vitest and use parseCLI ([#&#8203;21890](https://togithub.com/nrwl/nx/pull/21890))
-   **vue:** fixing vue and nuxt welcome templates ([#&#8203;21792](https://togithub.com/nrwl/nx/pull/21792))
-   **vue:** tailwind generator ignoring styleSheet option ([#&#8203;21840](https://togithub.com/nrwl/nx/pull/21840))
-   **webpack:** resolve relative path for assets inputs ([#&#8203;21822](https://togithub.com/nrwl/nx/pull/21822))
-   **webpack:** correctly handle paranthesis in PostCSS in url ([#&#8203;21884](https://togithub.com/nrwl/nx/pull/21884))
-   **webpack:** surface original error when remotes fail to start ([#&#8203;21919](https://togithub.com/nrwl/nx/pull/21919))

##### ❤️  Thank You

-   Austin Fahsl [@&#8203;fahslaj](https://togithub.com/fahslaj)
-   Colum Ferry [@&#8203;Coly010](https://togithub.com/Coly010)
-   Craigory Coppola [@&#8203;AgentEnder](https://togithub.com/AgentEnder)
-   Dan Roujinsky
-   Edouard Bozon [@&#8203;edbzn](https://togithub.com/edbzn)
-   Emily Xiong [@&#8203;xiongemi](https://togithub.com/xiongemi)
-   Jack Hsu [@&#8203;jaysoo](https://togithub.com/jaysoo)
-   Jonathan Cammisuli
-   Julian Martin
-   Katerina Skroumpelou [@&#8203;mandarini](https://togithub.com/mandarini)
-   Leosvel Pérez Espinosa [@&#8203;leosvelperez](https://togithub.com/leosvelperez)
-   Miroslav Jonaš [@&#8203;meeroslav](https://togithub.com/meeroslav)
-   Nicholas Cunningham [@&#8203;ndcunningham](https://togithub.com/ndcunningham)
-   Remco Krams
-   Steven Nance [@&#8203;llwt](https://togithub.com/llwt)
-   Tine Kondo [@&#8203;tinesoft](https://togithub.com/tinesoft)
-   Vadim Goy
-   Viktor Pöntinen
-   Zachary DeRose [@&#8203;ZackDeRose](https://togithub.com/ZackDeRose)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/toeverything/AFFiNE).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yMDAuMCIsInVwZGF0ZWRJblZlciI6IjM3LjIxMi4wIiwidGFyZ2V0QnJhbmNoIjoiY2FuYXJ5In0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants