Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server assets files #23

Closed
iMrDJAi opened this issue Jan 30, 2023 · 25 comments
Closed

Server assets files #23

iMrDJAi opened this issue Jan 30, 2023 · 25 comments

Comments

@iMrDJAi
Copy link

iMrDJAi commented Jan 30, 2023

Hello again,

I'm using some files from my server code (with fs.readFile(path) for example), and these aren't included in the production build.
My first attempt was the ?url suffix, but it's broken. await import('./file.png?url').default returns a valid path (/assets/file.0df35ef5.png), but it doesn't copy the file itself to dist/server/assets but instead it generates a js file (maps_icon.2a098fde.js) with the following content:

"use strict";
Object.defineProperty(exports, Symbol.toStringTag, { value: "Module" });
const maps_icon = "/assets/maps_icon.0df35ef5.png";
exports.default = maps_icon;

And unlike client side, I cannot use /public directory as they are internal (private), not sure what's the alternative. What is the recommended way to handle sever assets in both dev and prod builds?

@cyco130
Copy link
Owner

cyco130 commented Jan 30, 2023

You're running into this Vite issue. Although it has been marked as closed by this PR, no new stable version has been released since. But 4.1 beta might have it. (update: I checked, it 4.1 beta does have this).

If you have a separate client build, one possible workaround is to make sure those files are referenced from the client build too. If not, you may have to introduce one, just to build the assets.

Otherwise you can try the beta and see if it works.

I'll leave this issue open because I have a use case for this feature too.

@iMrDJAi
Copy link
Author

iMrDJAi commented Jan 30, 2023

I'll try 4.1 beta and see if it works, I'll keep you updated!

@cyco130
Copy link
Owner

cyco130 commented Jan 30, 2023

The option you're looking for is build.ssrEmitAssets, I think. I'm curious about how it works too!

@iMrDJAi
Copy link
Author

iMrDJAi commented Jan 31, 2023

Upgrading to Vite v4 was much complicated than I expected, some dependencies are no longer functional. This is due to vitejs/vite#11101. I'll try applying the changes to Vite v3 manually and tell you the result.

@cyco130
Copy link
Owner

cyco130 commented Jan 31, 2023

Oh, I don't think you'll receive much support in the future if you stick with CJS, unfortunately.

@iMrDJAi
Copy link
Author

iMrDJAi commented Jan 31, 2023

@cyco130 I've attempted to migrate my project to ESM more than once, but it didn't go well.

My first deal breaker with ESM is the need of adding .js extensions to relative import paths despite the imported file in the source code has the .ts/.tsx. While I'm okay with adding extensions, this behavior doesn't make sense to me, it just makes the code harder to read.

Another issue was the circular dependencies, they are not that hard to resolve but the process would take a very long time to complete.

And the final nail in the coffin, the incompatibility of some dependencies with ESM, and there is not much I could do regarding that, this is the main reason why I've aborted the whole thing.

Unless there will be an environment that may support both CJS and ESM, the world isn't ready for ESM yet. We can't just throw tons of perfectly working but unmaintained CJS libraries.

@iMrDJAi
Copy link
Author

iMrDJAi commented Jan 31, 2023

@cyco130 I have applied the changes to Vite v3 and I confirm that it worked!

@cyco130
Copy link
Owner

cyco130 commented Jan 31, 2023

@cyco130 I have applied the changes to Vite v3 and I confirm that it worked!

Great to hear. In that case I'm closing this issue since it's not really vavite related.

My first deal breaker with ESM is the need of adding .js extensions

If you're using vavite (or vite itself) for your entire project, you can keep using moduleResolution: "node" is your tsconfig.json so you don't have to add the extension. Alternatively, you can switch to bundler resolution mode in TypeScript 5 beta.

Another issue was the circular dependencies

Can't comment on this without understanding the exact issue. ESM allows partially circular deps and they are also problematic in CJS.

the incompatibility of some dependencies with ESM

I have a Vite plugin for resolving the biggest one. Other incompatibilities can usually be resolved by tweaking ssr.optimizeDeps and ssr.noExternal in Vite config. But I know it's now always guaranteed.

Good luck!

@cyco130 cyco130 closed this as completed Jan 31, 2023
@iMrDJAi
Copy link
Author

iMrDJAi commented Jan 31, 2023

@cyco130 Thank you for your time! vite-plugin-cjs-interop seems to be a solution for the Vite v4 issue, I'll give it a try!

Edit: Unfortunately, the plugin didn't help either, a library I use (linkedom) didn't work.

Edit2: I ended up using patch-package.

@brillout
Copy link

brillout commented Mar 3, 2023

Edit2: I ended up using patch-package.

What's the patch you ended up applying? That would be very helpful for us to know as it will give us further insights into ESM/CJS compatibility problems and we may also be able to apply your patch by default.

@iMrDJAi
Copy link
Author

iMrDJAi commented Mar 3, 2023

@brillout I used patch-package against Vite@3, I applied the changes introduced in vitejs/vite#11430.
Still using CJS. 😕

@cyco130
Copy link
Owner

cyco130 commented Mar 5, 2023

Edit: Unfortunately, the plugin didn't help either, a library I use (linkedom) didn't work.

linkedom has a correctly packaged ESM build (I'm looking at version 0.14.22). It seems to work fine for me. vite-plugin-cjs-interop won't work on it because it's not CommonJS to begin with.

@iMrDJAi
Copy link
Author

iMrDJAi commented May 14, 2023

@cyco130 @brillout I began the migration again, I switched from "module": "commonjs" to es2020 and I'm getting bunch of random errors, some of them are from vavite itself:

components/Carousel.tsx:3:62 - error TS7016: Could not find a declaration file for module 'swiper'. 'C:/Users/name/Desktop/project/node_modules/swiper/swiper.esm.js' implicitly has an 'any' type.
  There are types at 'C:/Users/name/Desktop/project/node_modules/swiper/swiper.d.ts', but this result could not be resolved when respecting package.json "exports". The 'swiper' library may need to update its package.json or typings.

3 import { Lazy, Navigation, Keyboard, FreeMode, Thumbs } from 'swiper'
                                                               ~~~~~~~~

pages/hotels/index.page.tsx:17:37 - error TS7016: Could not find a declaration file for module '@splidejs/react-splide'. 'C:/Users/name/Desktop/project/node_modules/@splidejs/react-splide/dist/js/react-splide.esm.js' implicitly has an 'any' type.
  There are types at 'C:/Users/name/Desktop/project/node_modules/@splidejs/react-splide/dist/types/index.d.ts', but this result could not be resolved when respecting package.json "exports". The '@splidejs/react-splide' library may need to update its package.json or typings.

17 import { Splide, SplideSlide } from '@splidejs/react-splide'
                                       ~~~~~~~~~~~~~~~~~~~~~~~~

server/index.ts:1:27 - error TS7016: Could not find a declaration file for module 'vavite/http-dev-server'. 'C:/Users/name/Desktop/project/node_modules/vavite/dist/http-dev-server.js' implicitly has an 'any' type.
  There are types at 'C:/Users/name/Desktop/project/node_modules/vavite/http-dev-server.d.ts', but this result could not be resolved when respecting package.json "exports". The 'vavite' library may need to update its package.json or typings.

1 import httpDevServer from 'vavite/http-dev-server'
                            ~~~~~~~~~~~~~~~~~~~~~~~~

suppliers/DOTW/DOTW.ts:21:34 - error TS2307: Cannot find module 'linkedom/types/esm/xml/document' or its corresponding type declarations.

21 import type { XMLDocument } from 'linkedom/types/esm/xml/document'
                                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

suppliers/DOTW/DOTW.ts:22:33 - error TS2307: Cannot find module 'linkedom/types/esm/mixin/parent-node' or its corresponding type declarations.

22 import type { NodeStruct } from 'linkedom/types/esm/mixin/parent-node'
                                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

vite.config.ts:1:23 - error TS2688: Cannot find type definition file for 'vavite/vite-config'.

1 /// <reference types='vavite/vite-config' />

This is what I was talking about all the time, ESM just scares me at this point. I'm usually left with little to no support, especially with unmaintained libraries. Thoughts?

@iMrDJAi
Copy link
Author

iMrDJAi commented May 14, 2023

For reference here is my complete tsconfig.json:

{
  "compilerOptions": {
    /* Visit https://aka.ms/tsconfig to read more about this file */

    /* Projects */
    "incremental": true,                              /* Save .tsbuildinfo files to allow for incremental compilation of projects. */
    // "composite": true,                                /* Enable constraints that allow a TypeScript project to be used with project references. */
    // "tsBuildInfoFile": "./.tsbuildinfo",              /* Specify the path to .tsbuildinfo incremental compilation file. */
    // "disableSourceOfProjectReferenceRedirect": true,  /* Disable preferring source files instead of declaration files when referencing composite projects. */
    // "disableSolutionSearching": true,                 /* Opt a project out of multi-project reference checking when editing. */
    // "disableReferencedProjectLoad": true,             /* Reduce the number of projects loaded automatically by TypeScript. */

    /* Language and Environment */
    "target": "es2017",                                  /* Set the JavaScript language version for emitted JavaScript and include compatible library declarations. */
    "lib": ["es2017", "dom"],                            /* Specify a set of bundled library declaration files that describe the target runtime environment. */
    "jsx": "react-jsx",                                  /* Specify what JSX code is generated. */
    // "experimentalDecorators": true,                   /* Enable experimental support for legacy experimental decorators. */
    // "emitDecoratorMetadata": true,                    /* Emit design-type metadata for decorated declarations in source files. */
    // "jsxFactory": "",                                 /* Specify the JSX factory function used when targeting React JSX emit, e.g. 'React.createElement' or 'h'. */
    // "jsxFragmentFactory": "",                         /* Specify the JSX Fragment reference used for fragments when targeting React JSX emit e.g. 'React.Fragment' or 'Fragment'. */
    // "jsxImportSource": "",                            /* Specify module specifier used to import the JSX factory functions when using 'jsx: react-jsx*'. */
    // "reactNamespace": "",                             /* Specify the object invoked for 'createElement'. This only applies when targeting 'react' JSX emit. */
    // "noLib": true,                                    /* Disable including any library files, including the default lib.d.ts. */
    "useDefineForClassFields": true,                     /* Emit ECMAScript-standard-compliant class fields. */
    // "moduleDetection": "auto",                        /* Control what method is used to detect module-format JS files. */

    /* Modules */
    "module": "es2020",                                  /* Specify what module code is generated. */
    // "rootDir": "./",                                  /* Specify the root folder within your source files. */
    "moduleResolution": "bundler",                       /* Specify how TypeScript looks up a file from a given module specifier. */
    // "baseUrl": "./",                                  /* Specify the base directory to resolve non-relative module names. */
    // "paths": {},                                      /* Specify a set of entries that re-map imports to additional lookup locations. */
    // "rootDirs": [],                                   /* Allow multiple folders to be treated as one when resolving modules. */
    "typeRoots": ["./types"],                            /* Specify multiple folders that act like './node_modules/@types'. */
    "types": ["vite/client"],                            /* Specify type package names to be included without being referenced in a source file. */
    // "allowUmdGlobalAccess": true,                     /* Allow accessing UMD globals from modules. */
    // "moduleSuffixes": [],                             /* List of file name suffixes to search when resolving a module. */
    "allowImportingTsExtensions": true,                  /* Allow imports to include TypeScript file extensions. Requires '--moduleResolution bundler' and either '--noEmit' or '--emitDeclarationOnly' to be set. */
    // "resolvePackageJsonExports": true,                /* Use the package.json 'exports' field when resolving package imports. */
    // "resolvePackageJsonImports": true,                /* Use the package.json 'imports' field when resolving imports. */
    // "customConditions": [],                           /* Conditions to set in addition to the resolver-specific defaults when resolving imports. */
    "resolveJsonModule": true,                           /* Enable importing .json files. */
    "allowArbitraryExtensions": true,                    /* Enable importing files with any extension, provided a declaration file is present. */
    // "noResolve": true,                                /* Disallow 'import's, 'require's or '<reference>'s from expanding the number of files TypeScript should add to a project. */

    /* JavaScript Support */
    "allowJs": true,                                     /* Allow JavaScript files to be a part of your program. Use the 'checkJS' option to get errors from these files. */
    // "checkJs": true,                                  /* Enable error reporting in type-checked JavaScript files. */
    // "maxNodeModuleJsDepth": 1,                        /* Specify the maximum folder depth used for checking JavaScript files from 'node_modules'. Only applicable with 'allowJs'. */

    /* Emit */
    // "declaration": true,                              /* Generate .d.ts files from TypeScript and JavaScript files in your project. */
    // "declarationMap": true,                           /* Create sourcemaps for d.ts files. */
    // "emitDeclarationOnly": true,                      /* Only output d.ts files and not JavaScript files. */
    // "sourceMap": true,                                /* Create source map files for emitted JavaScript files. */
    // "inlineSourceMap": true,                          /* Include sourcemap files inside the emitted JavaScript. */
    // "outFile": "./",                                  /* Specify a file that bundles all outputs into one JavaScript file. If 'declaration' is true, also designates a file that bundles all .d.ts output. */
    // "outDir": "./",                                   /* Specify an output folder for all emitted files. */
    // "removeComments": true,                           /* Disable emitting comments. */
    "noEmit": true,                                      /* Disable emitting files from a compilation. */
    // "importHelpers": true,                            /* Allow importing helper functions from tslib once per project, instead of including them per-file. */
    // "importsNotUsedAsValues": "remove",               /* Specify emit/checking behavior for imports that are only used for types. */
    // "downlevelIteration": true,                       /* Emit more compliant, but verbose and less performant JavaScript for iteration. */
    // "sourceRoot": "",                                 /* Specify the root path for debuggers to find the reference source code. */
    // "mapRoot": "",                                    /* Specify the location where debugger should locate map files instead of generated locations. */
    // "inlineSources": true,                            /* Include source code in the sourcemaps inside the emitted JavaScript. */
    // "emitBOM": true,                                  /* Emit a UTF-8 Byte Order Mark (BOM) in the beginning of output files. */
    // "newLine": "crlf",                                /* Set the newline character for emitting files. */
    // "stripInternal": true,                            /* Disable emitting declarations that have '@internal' in their JSDoc comments. */
    // "noEmitHelpers": true,                            /* Disable generating custom helper functions like '__extends' in compiled output. */
    // "noEmitOnError": true,                            /* Disable emitting files if any type checking errors are reported. */
    // "preserveConstEnums": true,                       /* Disable erasing 'const enum' declarations in generated code. */
    // "declarationDir": "./",                           /* Specify the output directory for generated declaration files. */
    // "preserveValueImports": true,                     /* Preserve unused imported values in the JavaScript output that would otherwise be removed. */

    /* Interop Constraints */
    "isolatedModules": true,                             /* Ensure that each file can be safely transpiled without relying on other imports. */
    // "verbatimModuleSyntax": true,                     /* Do not transform or elide any imports or exports not marked as type-only, ensuring they are written in the output file's format based on the 'module' setting. */
    "allowSyntheticDefaultImports": true,                /* Allow 'import x from y' when a module doesn't have a default export. */
    "esModuleInterop": true,                             /* Emit additional JavaScript to ease support for importing CommonJS modules. This enables 'allowSyntheticDefaultImports' for type compatibility. */
    // "preserveSymlinks": true,                         /* Disable resolving symlinks to their realpath. This correlates to the same flag in node. */
    "forceConsistentCasingInFileNames": true,            /* Ensure that casing is correct in imports. */

    /* Type Checking */
    "strict": true,                                      /* Enable all strict type-checking options. */
    "noImplicitAny": true,                               /* Enable error reporting for expressions and declarations with an implied 'any' type. */
    "strictNullChecks": true,                            /* When type checking, take into account 'null' and 'undefined'. */
    "strictFunctionTypes": true,                         /* When assigning functions, check to ensure parameters and the return values are subtype-compatible. */
    // "strictBindCallApply": true,                      /* Check that the arguments for 'bind', 'call', and 'apply' methods match the original function. */
    // "strictPropertyInitialization": true,             /* Check for class properties that are declared but not set in the constructor. */
    "noImplicitThis": true,                              /* Enable error reporting when 'this' is given the type 'any'. */
    // "useUnknownInCatchVariables": true,               /* Default catch clause variables as 'unknown' instead of 'any'. */
    // "alwaysStrict": true,                             /* Ensure 'use strict' is always emitted. */
    // "noUnusedLocals": true,                           /* Enable error reporting when local variables aren't read. */
    // "noUnusedParameters": true,                       /* Raise an error when a function parameter isn't read. */
    // "exactOptionalPropertyTypes": true,               /* Interpret optional property types as written, rather than adding 'undefined'. */
    // "noImplicitReturns": true,                        /* Enable error reporting for codepaths that do not explicitly return in a function. */
    // "noFallthroughCasesInSwitch": true,               /* Enable error reporting for fallthrough cases in switch statements. */
    // "noUncheckedIndexedAccess": true,                 /* Add 'undefined' to a type when accessed using an index. */
    // "noImplicitOverride": true,                       /* Ensure overriding members in derived classes are marked with an override modifier. */
    // "noPropertyAccessFromIndexSignature": true,       /* Enforces using indexed accessors for keys declared using an indexed type. */
    // "allowUnusedLabels": true,                        /* Disable error reporting for unused labels. */
    "allowUnreachableCode": true,                        /* Disable error reporting for unreachable code. */

    /* Completeness */
    // "skipDefaultLibCheck": true,                      /* Skip type checking .d.ts files that are included with TypeScript. */
    "skipLibCheck": true                                 /* Skip type checking all .d.ts files. */
  }
}

@iMrDJAi
Copy link
Author

iMrDJAi commented May 14, 2023

Update: fixed by setting resolvePackageJsonExports to false. Not sure why and how this has become the standard.

@brillout
Copy link

@iMrDJAi Thanks for circling back on this. Makes sense. Vavite could (I'd argue should :-)) support setups regardless of the resolvePackageJsonExports value.

@iMrDJAi
Copy link
Author

iMrDJAi commented May 15, 2023

@iMrDJAi Thanks for circling back on this. Makes sense. Vavite could (I'd argue should :-)) support setups regardless of the resolvePackageJsonExports value.

@brillout Other things should be taken into consideration:

Enabling --experimental-specifier-resolution=node flag when running the server build to automatically redirect library/something/ to library/something/index.js:

"server": "cross-env NODE_ENV=production node --experimental-specifier-resolution=node dist/server/index.mjs"

Now, I'm stuck with this issue #16 about CJS modules incompatibilities. Tracing CJS dependencies to add them to the vite-plugin-cjs-interop configuration is pain. Couldn't get a working production build so I decided to revert to CJS again till I find a more concrete solution. Perhaps, a full server build (#24) would solve the issue.

@iMrDJAi
Copy link
Author

iMrDJAi commented Jul 9, 2023

@brillout @cyco130 Seems like the new Node ESM loader implemented by Vavite has fixed compatibility issues between prod and dev, but in the other way around. While Vite's own loader provides support for both ESM/CJS imports that are not normally supported by Node, it's more important to have a consistent behavior for imports between dev and prod builds.

There still one problem, fake ESM exports, for example, this one from next-auth (next-auth/providers/credentials.js):

"use strict";

Object.defineProperty(exports, "__esModule", {
  value: true
});
exports.default = Credentials;

function Credentials(options) {
  return {
    id: "credentials",
    name: "Credentials",
    type: "credentials",
    credentials: {},
    authorize: () => null,
    options
  };
}
import CredentialsProvider from 'next-auth/providers/credentials'
console.log(CredentialsProvider) // { default: [Function: Credentials] }
// TypeError: CredentialsProvider is not a function

Normally, CredentialsProvider is a function, but here it returns an object with default prop. And while I can simply do CredentialsProvider.default, TypeScript won't be happy about it:

import CredentialsProvider from 'next-auth/providers/credentials'
console.log(CredentialsProvider.default) // [Function: Credentials]
// Property 'default' does not exist on type '<C extends Record<string, CredentialInput> = Record<string, CredentialInput>>(options: UserCredentialsConfig<C>) => CredentialsConfig<...>'

Now, we have inconsistent types and imports ¯\_(ツ)_/¯.

Any thoughts?

@brillout
Copy link

It's a Vite thing and AFAICT it's unrelated to vavite. See https://github.com/cyco130/vite-plugin-cjs-interop. (it's on my radar to find a workaround that works automatically.)

@cyco130
Copy link
Owner

cyco130 commented Jul 10, 2023

Seems like the new Node ESM loader implemented by Vavite has fixed compatibility issues between prod and dev, but in the other way around.

That's true. Mainly because it's not a vavite problem. It's not even a vite problem. It's Node that decided not to support fake ESM. Like @brillout said, the best fix is vite-plugin-cjs-interop but it requires you to manually list all deps that needs to be fixed. Someone expressed interest in making it automatic, so maybe we'll have a good fix soon, we'll see.

@brillout
Copy link

Someone expressed interest in making it automatic

Was is it me or was it someone else?

@cyco130
Copy link
Owner

cyco130 commented Jul 10, 2023

Was is it me or was it someone else?

Someone else from the Vite Discord.

@iMrDJAi
Copy link
Author

iMrDJAi commented Jul 10, 2023

@cyco130 @brillout That's actually me on Discord. I took a look once on the code of vite-plugin-cjs-interop and it seems simple. If I find a way to analyze and list the affected dependencies I can pass the array to it, and it will do the magic.

It's Node that decided not to support fake ESM.

I heard that Bun supports mixed CJS/ESM out of the box, but not sure how strict it is when it's about handling ESM. Has anyone tried it?

@cyco130
Copy link
Owner

cyco130 commented Jul 10, 2023

That's actually me on Discord

Sorry I didn't pay attention to the handle 😅

@brillout
Copy link

it seems simple. If I find a way to analyze and list the affected dependencies I can pass the array to it, and it will do the magic.

That's really neat if we can get that. I wonder, how do you detect problematic deps?

I heard that Bun supports mixed CJS/ESM out of the box, but not sure how strict it is when it's about handling ESM. Has anyone tried it?

Yea Jarred told me that as well. Didn't try yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants