Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

avoid writing the temporary file when evaluating the config file #9470

Open
4 tasks done
mihaon opened this issue Jul 31, 2022 · 26 comments · Fixed by #13269
Open
4 tasks done

avoid writing the temporary file when evaluating the config file #9470

mihaon opened this issue Jul 31, 2022 · 26 comments · Fixed by #13269
Labels
enhancement New feature or request

Comments

@mihaon
Copy link

mihaon commented Jul 31, 2022

Description

I run vite inside a docker container for security reasons.

My docker run looks like:

docker container run -d --rm --name "myApp" \
	-v "~/myApp:/var/www/myApp:ro" \
	-v "~/myApp/tmp:/var/www/myApp/tmp:rw" \
	-v "~/myApp/logs:/var/www/myApp/logs:rw" \
	-v "~/myApp/package-lock.json:/var/www/myApp/package-lock.json:rw" \
	"myApp:latest"

As you can see, the application code files are read-only and they can't be changed from inside the docker container (for example, some mistake like rm -rf * can't remove all my code files).

But since version 3.0.1, vite build creates the vite.config.js.mjs file in the root directory, and since version 3.0.3 it creates a file with a random name like vite.config.js.timestamp-1659277944620.mjs.

Because my application root is the read-only, then docker container exec -i "myApp" bash -l -c 'npm run build' fails with error:

failed to load config from /var/www/myApp/vite.config.js
error during build:
Error: EROFS: read-only file system, open '/var/www/myApp/vite.config.js.timestamp-1659277944620.mjs'

Suggested solution

Write file only to explicitly set directories.

Alternative

No response

Additional context

No response

Validations

@mihaon mihaon changed the title Vite should not write a files outside of "build.outDir" vite build should not write a files outside of "build.outDir" Jul 31, 2022
@mihaon mihaon changed the title vite build should not write a files outside of "build.outDir" vite build should not write a files outside of build.outDir Jul 31, 2022
@bluwy
Copy link
Member

bluwy commented Jul 31, 2022

vite.config.js.timestamp-1659277944620.mjs is a file bundled by Vite to read the config file. It happens for TS by default in Vite 3.0.0, and expanded to do so for JS too in Vite 3.0.1.

It doesn't generate to build.outDir because the import paths would then be different. I'm not sure how this can be fixed in Vite, but the Docker setup may need to relax a bit for now, until something like #5370 (comment) is implemented.

@mihaon
Copy link
Author

mihaon commented Jul 31, 2022

But why does Vite create this file for the ESM config?

Look at my vite.config.js file:

import { sveltekit } from '@sveltejs/kit/vite'
const rootDir = '/var/www/myApp'
export default {
	plugins: [sveltekit()],
	resolve: {
		alias: {
			'#root': rootDir,
		},
	},
}

And look at the temporary vite.config.js.timestamp-1659288603886.mjs file:

// vite.config.js
import { sveltekit } from "@sveltejs/kit/vite";
var rootDir = "/var/www/myApp";
var vite_config_default = {
  plugins: [sveltekit()],
  resolve: {
    alias: {
      "#root": rootDir
    }
  }
};
export {
  vite_config_default as default
};
//# sourceMappingURL=data:application/json;base64,...

There are no significant differences. Maybe Vite should only create the temporary file when needed?

@mihaon mihaon changed the title vite build should not write a files outside of build.outDir vite build should not write a files outside of explicitly set directories (build.outDir, etc.) Jul 31, 2022
@lyleaf
Copy link

lyleaf commented Sep 14, 2022

I got the se problem. I'm trying to put vite project into Google app engine which also doesn't allow write. It's throwing me error because vite try to generate the timestamped temp file.

@leobarcellos
Copy link

I'm also having this problem deploying in a Aws Lambda

@lyleaf
Copy link

lyleaf commented Sep 22, 2022

I solved it by just using a python yaml instead of a nodejs yaml, as mine is a static website.

@silverwind
Copy link

silverwind commented Mar 29, 2023

These vite.config.js.timestamp-1680094124134.mjs files also cause race conditions with make's dependency tracking because the file only exists momentarily and if it's removed after the make invocation, make will terminate with such errors:

make[1]: *** No rule to make target `vite.config.js.timestamp-1680094124134.mjs', needed by `build'.  Stop.

Can this file be removed or at least be moved to vite's temp dir node_modules/.vite?

@ivolkoff
Copy link

ivolkoff commented Apr 4, 2023

Files vite.config.js.timestamp make lerna mistakenly think that there are changes and rubuild

This file needs to be removed somewhere from the parent directory

lerna watch -- echo \$LERNA_PACKAGE_NAME \$LERNA_FILE_CHANGES
lerna notice cli v6.6.1
lerna info versioning independent
lerna info watch Executing command "echo $LERNA_PACKAGE_NAME $LERNA_FILE_CHANGES" on changes in 6 packages.
@app/helpers packages/_helpers/src/index.ts
@app/helpers packages/_helpers/vite.config.ts.timestamp-1680605632220.mjs
@app/helpers packages/_helpers/vite.config.ts.timestamp-1680605632220.mjs
@app/helpers packages/_helpers/vite.config.ts.timestamp-1680605636113.mjs
@app/helpers packages/_helpers/vite.config.ts.timestamp-1680605636113.mjs
@app/helpers packages/_helpers/vite.config.ts.timestamp-1680605639994.mjs
@app/helpers packages/_helpers/vite.config.ts.timestamp-1680605639994.mjs
@app/helpers packages/_helpers/vite.config.ts.timestamp-1680605644195.mjs
@app/helpers packages/_helpers/vite.config.ts.timestamp-1680605644195.mjs

@ducan-ne
Copy link

ducan-ne commented Apr 9, 2023

I get this file randomly and need to remove it by hand, quite annoying 😅

@8ctopotamus
Copy link

8ctopotamus commented Apr 17, 2023

This is happening to me too. I just threw this in my .gitignore:

vite.config.ts.timestamp-*.mjs

@silverwind
Copy link

silverwind commented Apr 21, 2023

I took a look a the hack, but it has no easy solution at least. The file must be at the same location for path resolution to work, which is the reason for the file's location.

The function already has access to the transpiled source which in theory could be ran as a string without temp files, but node offers no suitable API. VM modules would be suitable but are behind a experimental flag since a long time now.

I think the hack solely exists to support Typescript config files, all other variations seem to work fine with node's ESM importer, so the comment above it seems to have become inaccurate over time.

@patak-dev
Copy link
Member

I think Vite should at least be able to write files to its configured cacheDir. It may still bring issues with the current setup of the OP, but I think it should at least relax the restriction to allow modifications to cacheDir (we'll need when optimizing dependencies during build time, or other future caches). IMO, we should have used this dir for vite.config.ts.timestamp-{x}.mjs. But this issue could now be closed if we merge #13269

@patak-dev patak-dev linked a pull request Jun 1, 2023 that will close this issue
9 tasks
@silverwind
Copy link

silverwind commented Jun 1, 2023

#13269 does not fully solve it imho. One solution might be to check NODE_OPTIONS and if it contains --experimental-vm-modules, skip creating the temp file and create it completely in-memory.

With that, people have at least an option to solve this issue of that temp file. It would require the config file to be ESM, but I think that is a given already.

Edit: seeing the code of #13269, If that base64 import works, it will actually solve it.

@prodkt
Copy link

prodkt commented Jun 16, 2023

Can someone link me to a fix? This has taken me down. Hours in and I need to just prune away this vite.config.ts.timestamp-1686938782284-10d99850f42b.mjs but no matter what command, pnpm, npm yarn or shell bash vsh it only wants to run this file, even though it doesn't seem to appear anywhere, it's not hidden either. It's just stuck on only wanting to run this somehow altered config.

@github-actions github-actions bot locked and limited conversation to collaborators Jul 1, 2023
@patak-dev patak-dev reopened this Jul 6, 2023
@vitejs vitejs unlocked this conversation Jul 6, 2023
@iencotech
Copy link

I am experiencing this issue when running vite in development mode in an ubuntu server where I cannot grant write permissions in the project folder to the user that is running it:

Error: EACCES: permission denied, open '/path/to/my-project/vite.config.ts.timestamp-1698242656962-64a6d009d3b8a.mjs

The development server cannot start because of this.

@robertjpayne
Copy link

Trying to use vite in a Swift Package Plugin and it sandboxes the disk which means it cannot run due to it attempting to write this file.

@moghwan
Copy link

moghwan commented Feb 12, 2024

This is happening to me too. I just threw this in my .gitignore:

vite.config.ts.timestamp-*.mjs

sadly didn't work for me

I am experiencing this issue when running vite in development mode in an ubuntu server where I cannot grant write permissions in the project folder to the user that is running it:

Error: EACCES: permission denied, open '/path/to/my-project/vite.config.ts.timestamp-1698242656962-64a6d009d3b8a.mjs

The development server cannot start because of this.

For docker fellows, tried two workarounds:

  • I restarted everything fresh after running docker-compose down -v
  • in my Dockerfile I found we're using a specific user to run npm run dev, commenting it out to be run with default user root did the job
...

# USER node

CMD ["npm","run","dev"]

other than that, it would be great if we could set a path for the vite.config.ts.timestamp-*.mjs file

@robertjpayne
Copy link

robertjpayne commented Feb 21, 2024

This is actually a really challenging issue to solve because the "bundled" config has to be evaluated in the same directory as the original file itself. I dug into it a bit further and it doesn't look like there is really any solution today without using experimental node features.

@patak-dev it would've/could've been worth leaving the base64 encoded data url import in as an option via environment variable flag or something. There's quite a lot of users (myself included) that want to use vite where the source structure is "read only". Though I understand this may just create more headache for the vite team as there will invariably be plugins that break using that env flag and then resulting in issues.

@silverwind
Copy link

silverwind commented Feb 21, 2024

How do other projects that support .config.ts load their config files? I can't imagine they have similar hacks.

@robertjpayne
Copy link

@silverwind what sort of hacks do you mean?

@silverwind
Copy link

silverwind commented Feb 21, 2024

By hack I mean the temp file creation. The comment says "we have to do a hack".

@robertjpayne
Copy link

robertjpayne commented Feb 21, 2024

Ok @patak-dev @silverwind I have a novel way of doing this without any nodejs experimental flags or issues with the resolver! I believe it works and I tested it against @cpojer's sample repo and it seems to work.

Node.js now has stable hooks for module resolution and module loading. What this means is we can register a hook that will look for a "magic" file name and when detected do something special.

Hooks run in an isolated worker thread and so we're still going to need to encode the full contents of the config file as a base64 string. Instead of using a data:text/javascript;base64, uri though we're going to do a full url that just has the base64 contents encoded into the filename portion of it.

The loadConfigFromBundledFile function would look something like:

if (isESM) {
  const fileBase = `${fileName}.virtual-0x0-base64-${Buffer.from(bundledCode).toString('base64')}`
  const fileUrl = pathToFileURL(fileBase)
  try {
    return (await import(fileUrl)).default
  } finally {
    throw new Error(`${e.message} at ${fileName}`)
  }
}

Without doing anything else though Node.js is going to complain so we need to register our hooks to ensure this specific file name will resolve and load:

const registerHooks = `
import { Buffer } from "node:buffer";

const pattern = /virtual-0x0-base64-(.*)$/;

export async function resolve(specifier, context, nextResolve) {
  const match = specifier.match(pattern);
  if (match) {
    return {
      format: 'module',
      shortCircuit: true,
      url: specifier
    };
  }
  return await nextResolve(specifier);
};

export async function load(url, context, nextLoad) {
  const match = url.match(pattern);
  if (match) {
    const data = Buffer.from(match[1], 'base64');
    return {
      format: 'module',
      shortCircuit: true,
      source: data
    };
  }
  return nextLoad(url);
};
`
register(`data:text/javascript,${registerHooks}`, import.meta.url);

The resolve hook looks for a match and tells Node.js that it's a module that should be loaded at the url specified and the loader hook matches the base64 and decodes it returning it.

Because it's still using a url, node will treat everything else as-if it was a plain ole file.

EDIT: 🤦🏻 I realised the comments above the code mention using loaders, they are not experimental anymore and are not for any version that Vite 5.x supports so is there any reason not to use them?

@silverwind
Copy link

silverwind commented Feb 21, 2024

Customization Hooks and module.register are "RC" stability in Node 20, so I think it could be a viable option, but to keep compatibility with node 18 and bun, it should at least still fall back to the old method.

@sapphi-red sapphi-red changed the title vite build should not write a files outside of explicitly set directories (build.outDir, etc.) avoid writing the temporary file when evaluating the config file Mar 1, 2024
@sapphi-red
Copy link
Member

@robertjpayne I think we can try the base64 approach again (#13269) if we solve the problems we faced in the past (#13267 (comment)). The former problem is fine now as Vite 5 doesn't support Node.js 17. The latter problem needs to be solved.

@emanueljg
Copy link

This is a very annoying quirk. Breaks read-only envionments completely, like running inside a Nix store path.

@electrocnic
Copy link

electrocnic commented Apr 29, 2024

EDIT: After removing the Vitest Extension from VS Code, the temp files are not longer popping up, in my case.

Original Message:
In my case, this only happens when I add any vitest related npm packages to my project.
They do not even need to be used, I just need to add them with npm i --save-dev vitest (or any other vitest related package), I see the temporary vite config files popping up.

The solution for our team might therefore be: Drop vitest completely and use e2e and cypress component tests.

@MichaelNotych
Copy link

MichaelNotych commented May 1, 2024

Are there any updates? Got similar problem
2024-05-01 15:23:40 > frontend@0.0.1 build
2024-05-01 15:23:40 > vite build
2024-05-01 15:23:40 failed to load config from /app/vite.config.ts
2024-05-01 15:23:40 error during build:
2024-05-01 15:23:40 Error: EACCES: permission denied, open '/app/vite.config.ts.timestamp-1714562620704-172dde0daee49.mjs'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.