feat: persistent cache plugin #14333

Akryum · 2023-09-08T23:40:17Z

Description

Continuation of #10671

Additional context

What is the purpose of this pull request?

Bug fix
New Feature
Documentation update
Other

Before submitting the PR, please make sure you do the following

Read the Contributing Guidelines.
Read the Pull Request Guidelines and follow the PR Title Convention.
Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
Ideally, include relevant tests that fail without this PR but pass with it.

stackblitz · 2023-09-08T23:40:19Z

Run & review this pull request in StackBlitz Codeflow.

Akryum · 2023-09-08T23:46:00Z

We would still need a way to handle plugin's load with side effects such as in svelte's plugin, see https://github.com/vitejs/vite/pull/10671/files#r1010031224

ArnaudBarre

Thanks for this new proposal which is very aligned with what @yyx990803 suggested.

High level review that requires approval from the rest of the team:

The callbacks should be calculated in advanced which will be better for perf
The key&read callbacks should be merged in one callback
- This simplify the code
- This don't impose a string key to the plugin authors
- This allow the key to be async
Because the read cb is async, to avoid to many empty ticks if there is no "cache plugin", the plugin container should expose an optional cb

Akryum · 2023-09-13T10:16:51Z

The callbacks should be calculated in advanced which will be better for perf

Isn't getSortedPlugins already cached?

benmccann · 2023-10-29T22:59:41Z

packages/plugin-persistent-cache/package.json

+    "cache"
+  ],
+  "main": "./dist/index.cjs",
+  "module": "./dist/index.mjs",


No need to include module. It will be ignored since you've defined exports

Suggested change

"module": "./dist/index.mjs",

benmccann · 2023-10-29T23:01:01Z

packages/plugin-persistent-cache/package.json

+    "vite-plugin",
+    "cache"
+  ],
+  "main": "./dist/index.cjs",


I would suggest adding "type": "module" and only distributing an ESM version since this is a new plugin. Vite is working on dropping the CJS versions of its builds - they're deprecated in v5 and will be removed in v6

benmccann · 2023-10-29T23:03:00Z

packages/plugin-persistent-cache/README.md

@@ -0,0 +1,18 @@
+# @vitejs/plugin-persistent-cache [![npm](https://img.shields.io/npm/v/@vitejs/plugin-persistent-cache.svg)](https://npmjs.com/package/@vitejs/plugin-persistent-cache)
+


this could use some more docs:

what exactly is being cached?

how is invalidation handled?

i'm assuming it applies to the results of other plugins as well?

does it apply in both dev and prod?

can you control what directory the info is persisted in?

is there anything you need to do to use this on GitHub Actions?

the options filled out below

packages/plugin-persistent-cache/package.json

benmccann · 2023-10-30T18:39:46Z

packages/plugin-persistent-cache/src/utils.ts

+import fs from 'node:fs'
+import path from 'node:path'
+
+export function getCodeHash(code: string): string {


this could probably just be called getHash. it will work regardless of whether the input is a "code" or not

benmccann · 2023-10-30T18:40:21Z

packages/plugin-persistent-cache/src/utils.ts

+  return createHash('sha1').update(code).digest('hex')
+}
+
+export function tryStatSync(file: string): fs.Stats | undefined {


Suggested change

export function tryStatSync(file: string): fs.Stats | undefined {

function tryStatSync(file: string): fs.Stats | undefined {

benmccann · 2023-10-30T18:41:52Z

packages/plugin-persistent-cache/src/utils.ts

+  }
+}
+
+export function lookupFile(


Suggested change

export function lookupFile(

/** returns the first matching file path of the given file names */

export function lookupFile(

benmccann · 2023-10-30T18:47:19Z

packages/plugin-persistent-cache/package.json

+      "require": "./dist/index.cjs"
+    }
+  },
+  "scripts": {


I'm guessing we need to copy the lint, format, and typecheck from the main Vite package to be here as well

benmccann · 2023-10-30T18:53:35Z

packages/vite/src/node/server/transformRequest.ts

-  const loadResult = await pluginContainer.load(id, { ssr })
+  let loadResult: LoadResult
+
+  const cacheLoadResult = await pluginContainer.serveLoadCacheRead({


it looks like this happens in both dev and build in which case I don't think we should call it "serve" because it's a bit confusing with Vite's "serve" command (https://vitejs.dev/config/#conditional-config)

maybe we could call it something like getFromLoadCache

benmccann · 2023-10-30T18:57:58Z

packages/plugin-persistent-cache/src/options.ts

+  ])
+  if (packageLockFile) {
+    cacheVersionFromFiles.push(packageLockFile)
+  }


should we throw an exception if no package lock file is found? I think that would only happen if the user were using a different package manager and the caching logic might not be safe in that case

benmccann · 2023-10-30T20:37:52Z

I don't know if this would be considered out-of-scope, but it would be nice to have a way to cache generated assets, as it can be extremely expensive. E.g. there are requests for caching assets in JonasKruckenberg/imagetools#522, sveltejs/kit#11043, and zerodevx/svelte-img#35.

In vite-imagetools we currently do the following:

const fileHandle = this.emitFile({ type: 'asset' })
metadata.src = `__VITE_ASSET__${fileHandle}__`

And actually, I wonder if this plugin would break for projects with vite-imagetools installed if we don't have such support because otherwise I assume it's going to be referring to files that don't exist.

benmccann · 2023-11-20T21:36:15Z

We would still need a way to handle plugin's load with side effects such as in svelte's plugin, see https://github.com/vitejs/vite/pull/10671/files#r1010031224

I don't think we should worry about supporting side effects generally. I'd imagine we could update vite-plugin-svelte to use these new persistent cache hooks rather than its own transient in memory cache implementation. @dominikg can correct me if this wouldn't be feasible.

However, I do think we should support the emitFile side-effect mentioned in my last comment since it's a core part of Vite/Rollup.

feat: persistent cache plugin

580175d

fix: missing debug in devDeps

1538e6d

Akryum mentioned this pull request Sep 8, 2023

feat: experimental server persistent caching #10671

Closed

11 tasks

Akryum added 3 commits September 9, 2023 01:56

fix: skip transform cache if skipped in load cache

6ad0aa4

feat: check if files exists

0106c50

fix: update deps metadata

9de9b61

ArnaudBarre reviewed Sep 11, 2023

View reviewed changes

Akryum added 2 commits September 11, 2023 17:32

fix: deps browser hash

ff1a92b

revert: check if files exist

42c4389

refactor: remove getKey hooks

5473b93

benmccann mentioned this pull request Oct 29, 2023

Cache generated images? JonasKruckenberg/imagetools#522

Closed

benmccann reviewed Oct 29, 2023

View reviewed changes

benmccann reviewed Oct 30, 2023

View reviewed changes

benmccann added 2 commits November 19, 2023 09:07

merge master

d2a9a5f

Update packages/plugin-persistent-cache/package.json

c187335

patak-dev added the p2-to-be-discussed Enhancement under consideration (priority) label Feb 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: persistent cache plugin #14333

feat: persistent cache plugin #14333

Akryum commented Sep 8, 2023

stackblitz bot commented Sep 8, 2023

Akryum commented Sep 8, 2023

ArnaudBarre left a comment •

edited

Akryum commented Sep 13, 2023

benmccann Oct 29, 2023

benmccann Oct 29, 2023

benmccann Oct 29, 2023 •

edited

benmccann Oct 30, 2023

benmccann Oct 30, 2023

benmccann Oct 30, 2023

benmccann Oct 30, 2023

benmccann Oct 30, 2023

benmccann Oct 30, 2023

benmccann commented Oct 30, 2023 •

edited

benmccann commented Nov 20, 2023 •

edited

		@@ -0,0 +1,18 @@
		# @vitejs/plugin-persistent-cache [![npm](https://img.shields.io/npm/v/@vitejs/plugin-persistent-cache.svg)](https://npmjs.com/package/@vitejs/plugin-persistent-cache)

	export function tryStatSync(file: string): fs.Stats \| undefined {
	function tryStatSync(file: string): fs.Stats \| undefined {

	export function lookupFile(
	/** returns the first matching file path of the given file names */
	export function lookupFile(

feat: persistent cache plugin #14333

Are you sure you want to change the base?

feat: persistent cache plugin #14333

Conversation

Akryum commented Sep 8, 2023

Description

Additional context

What is the purpose of this pull request?

Before submitting the PR, please make sure you do the following

stackblitz bot commented Sep 8, 2023

Akryum commented Sep 8, 2023

ArnaudBarre left a comment • edited

Choose a reason for hiding this comment

Akryum commented Sep 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benmccann Oct 29, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benmccann commented Oct 30, 2023 • edited

benmccann commented Nov 20, 2023 • edited

ArnaudBarre left a comment •

edited

benmccann Oct 29, 2023 •

edited

benmccann commented Oct 30, 2023 •

edited

benmccann commented Nov 20, 2023 •

edited