fix: crawl dependencies

[benmccann]

Description

Partially addresses #3024. This allows you to use CJS dependencies on the client-side. The server-side is still broken. I haven't looked into the cause for that yet, but that will probably involve larger changes since SSR is further away from being stable

Additional context

Most dependencies aren't pre-bundled today, which causes lots of issues. This is the top cause of user difficulties in SvelteKit today. Basically the only way you can use SvelteKit is by manually converting all your transitive dependencies to ESM. That can be quite hard to do when you depend on numerous npm modules. I've been trying to convert svelte.dev to SvelteKit and haven't been able to yet. I've hit four or five issues with dependencies and am still working through them and am not sure I can get the necessary changes into some of the libraries we use.

I don't understand if there's a reason the code was written the way it was initially. I didn't see any obvious reason for the current logic while doing a git blame. The docs say:

CommonJS and UMD compatibility: During development, Vite's dev serves all code as native ESM. Therefore, Vite must convert dependencies that are shipped as CommonJS or UMD into ESM first.

Vite today does not adhere to what was written in the docs because a large fraction of code is not served as native ESM without a change like the one in this PR

---

What is the purpose of this pull request?

• Bug fix
• New Feature
• Documentation update
• Other

Before submitting the PR, please make sure you do the following

• Read the Contributing Guidelines.
• Read the Pull Request Guidelines and follow the Commit Convention.
• Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
• Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
• Ideally, include relevant tests that fail without this PR but pass with it.

[benmccann]

Closing this pending further discussion

[stephenh]

@benmccann FWIW random comment from the peanut gallery (I'm subscribed to #3024), but in trying to convert a large create-react-app over to vitejs, I was similarly very confused by the mismatch between "the docs say that all dependencies are ESM-ified" ... but it doesn't seem to actually do that, at least transitively.

Really appreciate the work you're doing here! Regardless of whether the fix is to make it magically transitive (which would be great from my naive viewpoint), or even just clarify what the current behavior actually is, so that if "you must manually ESM-ify deps" (or deps that have transitively have CJS deps, or what not) is the only supported approach, then it's at least clear that's what we'd need to do.