Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: experimental server persistent caching #10671

Closed
wants to merge 59 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
14bde2b
feat: experimental server persistent caching
Akryum Oct 27, 2022
c309eed
fix: check if config is nullish
Akryum Oct 27, 2022
57f081d
fix: don't cache modules that contains 'import.meta.glob'
Akryum Oct 27, 2022
ce26960
feat: exclude
Akryum Oct 27, 2022
12216c3
fix: add id in cache key
Akryum Oct 27, 2022
a8f8d28
refactor: includedInPersistentCache
Akryum Oct 28, 2022
fc73790
fix: cache loadResult to support svelte
Akryum Oct 28, 2022
061e645
fix: resolve `versionFromFiles` path from `resolvedRoot`
Akryum Oct 28, 2022
d128365
fix: not waiting for clearing the cache files
Akryum Oct 28, 2022
138afc0
feat: clearing cache log
Akryum Oct 28, 2022
9767372
feat: clearer generated cacheVersion
Akryum Oct 28, 2022
228670c
feat: auto include tsconfig in versionFromFiles
Akryum Oct 28, 2022
d45302c
refactor: move resolve options from config to persistentCache
Akryum Oct 28, 2022
f4ff16b
feat: save request id in manifest
Akryum Oct 29, 2022
8c512e6
fix: manifest write debounce
Akryum Oct 29, 2022
5750ddb
feat: cleanup stale cache files
Akryum Oct 29, 2022
8c86c75
feat: cached modules HMR support
Akryum Oct 29, 2022
830b5fb
fix: disable cache file cleanup for now
Akryum Oct 29, 2022
0410cd3
fix: state cache files cleanup
Akryum Oct 29, 2022
f4b2e5e
feat: store url too
Akryum Nov 1, 2022
8dd8dd7
refactor: simplify resolvePersistentCacheOptions
Akryum Nov 1, 2022
967d8cd
refactor: remove cleanup
Akryum Nov 1, 2022
6b32599
refactor: debug log
Akryum Nov 1, 2022
d9695fb
fix: wording
Akryum Nov 1, 2022
59fcf27
perf: readFileSync manifest
Akryum Nov 1, 2022
60663a8
refactor: change cache dir location under .vite
Akryum Nov 1, 2022
662231b
refactor: use isDefined
Akryum Nov 1, 2022
f895075
chore: more comments
Akryum Nov 1, 2022
10a06d5
fix: ignore code containing import.meta.env for now
Akryum Nov 1, 2022
338e3e7
refactor: rename to serverPersistentCache
Akryum Nov 1, 2022
5f27f38
refactor: isFullCacheEntry
Akryum Nov 1, 2022
eb6b896
refactor: use resolved config code hash
Akryum Nov 1, 2022
9b2ecad
Merge remote-tracking branch 'refs/remotes/origin/main'
Akryum Nov 1, 2022
b045081
fix: build
Akryum Nov 1, 2022
6862e70
feat: exclude optimized deps and vite client
Akryum Nov 1, 2022
21462a4
revert: const resolvedConfigFile
Akryum Nov 1, 2022
31efe08
feat: remove cacheDir option
Akryum Nov 1, 2022
1ca741b
revert: ignore import.meta.env
Akryum Nov 1, 2022
8a38f30
feat: include define in cache version
Akryum Nov 1, 2022
ddf9b65
fix: correctly exclude vite client
Akryum Nov 1, 2022
362202d
fix: remove ?v from importedModules
Akryum Nov 1, 2022
8b90781
fix: rewrite optimized deps imports
Akryum Nov 2, 2022
69df6ad
fix: remove warning log
Akryum Nov 2, 2022
e4d0ec8
feat: include env in cache version
Akryum Nov 2, 2022
71f90e9
refactor: cleanup resolvePersistentCacheOptions
Akryum Nov 2, 2022
a7ae200
refactor: extract cache compute and manifest fns
Akryum Nov 3, 2022
03a3971
fix: optimized deps version query rewrite
Akryum Nov 3, 2022
65745f8
refactor: ResolvedExperimentalOptions
Akryum Nov 3, 2022
7004101
revert: bring back cacheDir option
Akryum Nov 3, 2022
add327f
fix: forgot to rename
Akryum Nov 3, 2022
c1d9f8b
chore: more debug logs
Akryum Nov 4, 2022
99a35ad
fix: temporary cache for patched modules
Akryum Nov 4, 2022
037972f
refactor: use a skip list instead of temp cache
Akryum Nov 4, 2022
466e479
chore: typo
Akryum Nov 4, 2022
f0da21f
fix: ssr code cache write
Akryum Nov 4, 2022
31eaebe
chore: add more debug log
Akryum Nov 4, 2022
84d7a70
refactor: getKey add ssr param
Akryum Nov 7, 2022
ed4d3b2
refactor: rename config to serverPersistentCache
Akryum Nov 7, 2022
373d02d
chore: config.cacheDir comment
Akryum Nov 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
72 changes: 68 additions & 4 deletions packages/vite/src/node/config.ts
Expand Up @@ -25,6 +25,7 @@ import {
createDebugger,
createFilter,
dynamicImport,
getCodeHash,
isBuiltin,
isExternalUrl,
isObject,
Expand Down Expand Up @@ -64,6 +65,7 @@ import type { PackageCache } from './packages'
import { loadEnv, resolveEnvPrefix } from './env'
import type { ResolvedSSROptions, SSROptions } from './ssr'
import { resolveSSROptions } from './ssr'
import { resolvePersistentCacheOptions } from './server/persistentCache'

const debug = createDebugger('vite:config')

Expand Down Expand Up @@ -292,6 +294,52 @@ export interface ExperimentalOptions {
* @default false
*/
hmrPartialAccept?: boolean
/**
* Enables persistent cache for the development server.
*
* @experimental
* @default null
*/
serverPersistentCache?: ServerPersistentCacheOptions | true
}

export interface ServerPersistentCacheOptions {
/**
* Enable or disable persistent cache.
*/
enabled?: boolean
/**
* Paths to files that should be taken into account when determining whether to clear the cache.
* By default will use the vite config file and your package lock file (for npm, yarn or pnpm).
*/
cacheVersionFromFiles?: string[]
/**
* Manual version string that should be taken into account when determining whether to clear the cache.
* Will be added to the hash of `cacheVersionFromFiles`.
*/
cacheVersion?: string
/**
* Exclude requests from being cached.
*/
exclude?: (url: string) => boolean
/**
* Name of the cache directory.
* If you have multiple vite servers running (e.g. Nuxt), you can use this to differentiate them.
* @default 'server-cache'
*/
cacheDir?: string
}

export interface ResolvedExperimentalOptions
extends Omit<ExperimentalOptions, 'serverPersistentCache'> {
serverPersistentCache: ResolvedServerPersistentCacheOptions | null
}

export interface ResolvedServerPersistentCacheOptions {
cacheDir: string
cacheVersionFromFiles: string[]
cacheVersion: string
exclude?: (url: string) => boolean
}

export interface LegacyOptions {
Expand All @@ -317,8 +365,12 @@ export interface InlineConfig extends UserConfig {
}

export type ResolvedConfig = Readonly<
Omit<UserConfig, 'plugins' | 'assetsInclude' | 'optimizeDeps' | 'worker'> & {
Omit<
UserConfig,
'plugins' | 'assetsInclude' | 'optimizeDeps' | 'worker' | 'experimental'
> & {
configFile: string | undefined
configFileHash: string | undefined
configFileDependencies: string[]
inlineConfig: InlineConfig
root: string
Expand Down Expand Up @@ -349,7 +401,7 @@ export type ResolvedConfig = Readonly<
packageCache: PackageCache
worker: ResolveWorkerOptions
appType: AppType
experimental: ExperimentalOptions
experimental: ResolvedExperimentalOptions
} & PluginHookUtils
>

Expand All @@ -373,6 +425,7 @@ export async function resolveConfig(
defaultMode = 'development'
): Promise<ResolvedConfig> {
let config = inlineConfig
let configFileHash: string | undefined
let configFileDependencies: string[] = []
let mode = inlineConfig.mode || defaultMode

Expand Down Expand Up @@ -404,6 +457,7 @@ export async function resolveConfig(
if (loadResult) {
config = mergeConfig(loadResult.config, config)
configFile = loadResult.path
configFileHash = loadResult.hash
configFileDependencies = loadResult.dependencies
}
}
Expand Down Expand Up @@ -616,8 +670,15 @@ export async function resolveConfig(
getSortedPluginHooks: undefined!
}

const serverPersistentCache = resolvePersistentCacheOptions({
config,
cacheDir,
resolvedRoot
})

const resolvedConfig: ResolvedConfig = {
configFile: configFile ? normalizePath(configFile) : undefined,
configFileHash,
configFileDependencies: configFileDependencies.map((name) =>
normalizePath(path.resolve(name))
),
Expand Down Expand Up @@ -663,7 +724,8 @@ export async function resolveConfig(
experimental: {
importGlobRestoreExtension: false,
hmrPartialAccept: false,
...config.experimental
...config.experimental,
serverPersistentCache
},
getSortedPlugins: undefined!,
getSortedPluginHooks: undefined!
Expand Down Expand Up @@ -865,6 +927,7 @@ export async function loadConfigFromFile(
path: string
config: UserConfig
dependencies: string[]
hash: string
} | null> {
const start = performance.now()
const getTime = () => `${(performance.now() - start).toFixed(2)}ms`
Expand Down Expand Up @@ -922,7 +985,8 @@ export async function loadConfigFromFile(
return {
path: normalizePath(resolvedPath),
config,
dependencies: bundled.dependencies
dependencies: bundled.dependencies,
hash: getCodeHash(bundled.code)
}
} catch (e) {
createLogger(logLevel).error(
Expand Down
9 changes: 9 additions & 0 deletions packages/vite/src/node/index.ts
Expand Up @@ -95,6 +95,15 @@ export type {
TransformResult
} from './server/transformRequest'
export type { HmrOptions, HmrContext } from './server/hmr'
export type {
PersistentCache,
PersistentCacheEntry,
PersistentSimpleCacheEntry,
PersistentFullCacheEntry,
PersistentCacheFile,
PersistentCacheManifest,
PersistentCacheResult
} from './server/persistentCache'

export type {
HMRPayload,
Expand Down
8 changes: 7 additions & 1 deletion packages/vite/src/node/optimizer/index.ts
Expand Up @@ -25,6 +25,7 @@ import {
} from '../utils'
import { transformWithEsbuild } from '../plugins/esbuild'
import { ESBUILD_MODULES_TARGET } from '../constants'
import type { ViteDevServer } from '../index'
import { esbuildCjsExternalPlugin, esbuildDepPlugin } from './esbuildDepPlugin'
import { scanImports } from './scan'
export {
Expand Down Expand Up @@ -450,7 +451,8 @@ export async function runOptimizeDeps(
resolvedConfig: ResolvedConfig,
depsInfo: Record<string, OptimizedDepInfo>,
ssr: boolean = resolvedConfig.command === 'build' &&
!!resolvedConfig.build.ssr
!!resolvedConfig.build.ssr,
server?: ViteDevServer
): Promise<DepOptimizationResult> {
const isBuild = resolvedConfig.command === 'build'
const config: ResolvedConfig = {
Expand Down Expand Up @@ -671,6 +673,10 @@ export async function runOptimizeDeps(
const dataPath = path.join(processingCacheDir, '_metadata.json')
writeFile(dataPath, stringifyDepsOptimizerMetadata(metadata, depsCacheDir))

if (server?._persistentCache) {
await server._persistentCache.updateDepsMetadata(metadata)
}

debug(`deps bundled in ${(performance.now() - start).toFixed(2)}ms`)

return processingResult
Expand Down
9 changes: 7 additions & 2 deletions packages/vite/src/node/optimizer/optimizer.ts
Expand Up @@ -230,7 +230,12 @@ async function createDepsOptimizer(
// run on the background, but we wait until crawling has ended
// to decide if we send this result to the browser or we need to
// do another optimize step
postScanOptimizationResult = runOptimizeDeps(config, knownDeps)
postScanOptimizationResult = runOptimizeDeps(
config,
knownDeps,
undefined,
server
)
}
} catch (e) {
logger.error(e.message)
Expand Down Expand Up @@ -271,7 +276,7 @@ async function createDepsOptimizer(

startNextDiscoveredBatch()

return await runOptimizeDeps(config, knownDeps)
return await runOptimizeDeps(config, knownDeps, undefined, server)
}

function prepareKnownDeps() {
Expand Down
9 changes: 8 additions & 1 deletion packages/vite/src/node/server/index.ts
Expand Up @@ -79,6 +79,8 @@ import { openBrowser } from './openBrowser'
import type { TransformOptions, TransformResult } from './transformRequest'
import { transformRequest } from './transformRequest'
import { searchForWorkspaceRoot } from './searchRoot'
import type { PersistentCache } from './persistentCache'
import { createPersistentCache } from './persistentCache'

export { searchForWorkspaceRoot } from './searchRoot'

Expand Down Expand Up @@ -301,6 +303,10 @@ export interface ViteDevServer {
* @internal
*/
_fsDenyGlob: Matcher
/**
* @internal
*/
_persistentCache: PersistentCache | null
}

export interface ResolvedServerUrls {
Expand Down Expand Up @@ -452,7 +458,8 @@ export async function createServer(
_importGlobMap: new Map(),
_forceOptimizeOnRestart: false,
_pendingRequests: new Map(),
_fsDenyGlob: picomatch(config.server.fs.deny, { matchBase: true })
_fsDenyGlob: picomatch(config.server.fs.deny, { matchBase: true }),
_persistentCache: await createPersistentCache(config)
}

server.transformIndexHtml = createDevHtmlTransformFn(server)
Expand Down