Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v3.0] New hashing algorithm that "fixes (nearly) everything" #4543

Merged
merged 33 commits into from Jul 5, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
53ebb6d
Initial new hashing idea
lukastaegert Apr 10, 2022
7b9a601
Simplify external import path generation
lukastaegert Apr 10, 2022
f6569c1
Use correct file names in chunk info
lukastaegert Apr 10, 2022
5b860a4
Implement first draft for hashing algorithm
lukastaegert Apr 12, 2022
8fe2adb
Remove active deprecations
lukastaegert Jun 13, 2022
8bc33c1
Reduce render parameters
lukastaegert Jun 15, 2022
86ded4a
Always scan all chunks for hashes
lukastaegert Jun 16, 2022
cf2ae17
Fix asset emission and remaining tests
lukastaegert Jun 19, 2022
10cd3c5
Reintroduce augmentChunkHash and get OutputChunk by converting Render…
lukastaegert Jun 19, 2022
806939c
Provide chunk graph in renderChunk
lukastaegert Jun 20, 2022
245036f
Handle hash collisions
lukastaegert Jun 20, 2022
b8c2090
Remove deprecated hacky asset emission
lukastaegert Jun 22, 2022
b81a687
Allow to configure hash sizes per file
lukastaegert Jun 24, 2022
46c89f9
Update documentation
lukastaegert Jun 24, 2022
b5fb37e
Extend tests
lukastaegert Jun 24, 2022
f89aaa9
Minor improvements
lukastaegert Jun 24, 2022
b14afef
Improve documentation about hashing
lukastaegert Jun 24, 2022
718465e
Replace hash in sourcemap file
lukastaegert Jun 25, 2022
9eaa9f3
Provide ChunkInfo in banner/footer/intro/outro
lukastaegert Jun 25, 2022
300222c
Extract hashing logic
lukastaegert Jun 26, 2022
23f59e3
Clean up hashing logic
lukastaegert Jun 26, 2022
b2ce403
Add ExternalChunk wrapper
lukastaegert Jun 28, 2022
4b8d93c
Store inputBase on Chunk
lukastaegert Jun 28, 2022
7d07060
Store snippets on Chunk
lukastaegert Jun 28, 2022
57e7be6
Align chunk interfaces
lukastaegert Jun 28, 2022
0b33abb
Reduce this. property access
lukastaegert Jun 28, 2022
e4a2bdf
Move dynamicImportFunction warning to options normalization
lukastaegert Jun 28, 2022
f61447d
Restructure rendering logic
lukastaegert Jun 30, 2022
6796d3a
Do not run on Node 10
lukastaegert Jul 1, 2022
f964499
Update documentation
lukastaegert Jul 1, 2022
e217f0d
Try to fix Windows tests
lukastaegert Jul 1, 2022
f05e4db
Improve coverage
lukastaegert Jul 2, 2022
4f1f6af
Remove graph background colors
lukastaegert Jul 3, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
56 changes: 35 additions & 21 deletions docs/05-plugin-development.md

Large diffs are not rendered by default.

21 changes: 14 additions & 7 deletions docs/999-big-list-of-options.md
Expand Up @@ -389,17 +389,24 @@ The pattern to use for naming custom emitted assets to include in the build outp

- `[extname]`: The file extension of the asset including a leading dot, e.g. `.css`.
- `[ext]`: The file extension without a leading dot, e.g. `css`.
- `[hash]`: A hash based on the name and content of the asset.
- `[hash]`: A hash based on the content of the asset. You can also set a specific hash length via e.g. `[hash:10]`.
- `[name]`: The file name of the asset excluding any extension.

Forward slashes `/` can be used to place files in sub-directories. When using a function, `assetInfo` is a reduced version of the one in [`generateBundle`](guide/en/#generatebundle) without the `fileName`. See also [`output.chunkFileNames`](guide/en/#outputchunkfilenames), [`output.entryFileNames`](guide/en/#outputentryfilenames).

#### output.banner/output.footer

Type: `string | (() => string | Promise<string>)`<br> CLI: `--banner`/`--footer <text>`
Type: `string | ((chunk: ChunkInfo) => string | Promise<string>)`<br> CLI: `--banner`/`--footer <text>`

A string to prepend/append to the bundle. You can also supply a function that returns a `Promise` that resolves to a `string` to generate it asynchronously (Note: `banner` and `footer` options will not break sourcemaps).

If you supply a function, `chunk` contains additional information about the chunk using the same `ChunkInfo` type as the [`generateBundle`](guide/en/#generatebundle) hook with the following differences:

- `code` and `map` are not set as the chunk has not been rendered yet.
- all referenced chunk file names that would contain hashes will contain hash placeholders instead. This includes `fileName`, `imports`, `importedBindings`, `dynamicImports` and `implicitlyLoadedBefore`. When you use such a placeholder file name or part of it in the code returned from this option, Rollup will replace the placeholder with the actual hash before `generateBundle`, making sure the hash reflects the actual content of the final generated chunk including all referenced file hashes.

`chunk` is mutable and changes applied in this hook will propagate to other plugins and to the generated bundle. That means if you add or remove imports or exports in this hook, you should update `imports`, `importedBindings` and/or `exports`.

```js
// rollup.config.js
export default {
Expand All @@ -421,10 +428,10 @@ Type: `string | ((chunkInfo: ChunkInfo) => string)`<br> CLI: `--chunkFileNames <
The pattern to use for naming shared chunks created when code-splitting, or a function that is called per chunk to return such a pattern. Patterns support the following placeholders:

- `[format]`: The rendering format defined in the output options, e.g. `es` or `cjs`.
- `[hash]`: A hash based on the content of the chunk and the content of all its dependencies.
- `[hash]`: A hash based only on the content of the final generated chunk, including transformations in [`renderChunk`](guide/en/#renderchunk) and any referenced file hashes. You can also set a specific hash length via e.g. `[hash:10]`.
- `[name]`: The name of the chunk. This can be explicitly set via the [`output.manualChunks`](guide/en/#outputmanualchunks) option or when the chunk is created by a plugin via [`this.emitFile`](guide/en/#thisemitfile). Otherwise, it will be derived from the chunk contents.

Forward slashes `/` can be used to place files in sub-directories. When using a function, `chunkInfo` is a reduced version of the one in [`generateBundle`](guide/en/#generatebundle) without properties that depend on file names. See also [`output.assetFileNames`](guide/en/#outputassetfilenames), [`output.entryFileNames`](guide/en/#outputentryfilenames).
Forward slashes `/` can be used to place files in sub-directories. When using a function, `chunkInfo` is a reduced version of the one in [`generateBundle`](guide/en/#generatebundle) without properties that depend on file names and no information about the rendered modules as rendering only happens after file names have been generated. You can however access a list of included `moduleIds`. See also [`output.assetFileNames`](guide/en/#outputassetfilenames), [`output.entryFileNames`](guide/en/#outputentryfilenames).

#### output.compact

Expand All @@ -439,10 +446,10 @@ Type: `string | ((chunkInfo: ChunkInfo) => string)`<br> CLI: `--entryFileNames <
The pattern to use for chunks created from entry points, or a function that is called per entry chunk to return such a pattern. Patterns support the following placeholders:

- `[format]`: The rendering format defined in the output options, e.g. `es` or `cjs`.
- `[hash]`: A hash based on the content of the entry point and the content of all its dependencies.
- `[hash]`: A hash based only on the content of the final generated entry chunk, including transformations in [`renderChunk`](guide/en/#renderchunk) and any referenced file hashes. You can also set a specific hash length via e.g. `[hash:10]`.
- `[name]`: The file name (without extension) of the entry point, unless the object form of input was used to define a different name.

Forward slashes `/` can be used to place files in sub-directories. When using a function, `chunkInfo` is a reduced version of the one in [`generateBundle`](guide/en/#generatebundle) without properties that depend on file names. See also [`output.assetFileNames`](guide/en/#outputassetfilenames), [`output.chunkFileNames`](guide/en/#outputchunkfilenames).
Forward slashes `/` can be used to place files in sub-directories. When using a function, `chunkInfo` is a reduced version of the one in [`generateBundle`](guide/en/#generatebundle) without properties that depend on file names and no information about the rendered modules as rendering only happens after file names have been generated. You can however access a list of included `moduleIds`. See also [`output.assetFileNames`](guide/en/#outputassetfilenames), [`output.chunkFileNames`](guide/en/#outputchunkfilenames).

This pattern will also be used when setting the [`output.preserveModules`](guide/en/#outputpreservemodules) option. Here a different set of placeholders is available, though:

Expand Down Expand Up @@ -809,7 +816,7 @@ There are some additional options that have an effect on the generated interop c

#### output.intro/output.outro

Type: `string | (() => string | Promise<string>)`<br> CLI: `--intro`/`--outro <text>`
Type: `string | ((chunk: ChunkInfo) => string | Promise<string>)`<br> CLI: `--intro`/`--outro <text>`

Similar to [`output.banner/output.footer`](guide/en/#outputbanneroutputfooter), except that the code goes _inside_ any format-specific wrapper.

Expand Down
1 change: 1 addition & 0 deletions docs/build-hooks.mmd
@@ -1,4 +1,5 @@
flowchart TB
classDef default fill:transparent, color:#000;
classDef hook-parallel fill:#ffb3b3,stroke:#000;
classDef hook-sequential fill:#ffd2b3,stroke:#000;
classDef hook-first fill:#fff2b3,stroke:#000;
Expand Down
64 changes: 42 additions & 22 deletions docs/output-generation-hooks.mmd
@@ -1,5 +1,5 @@
flowchart TB
classDef default fill:#fff;
classDef default fill:transparent, color:#000;
classDef hook-parallel fill:#ffb3b3,stroke:#000;
classDef hook-sequential fill:#ffd2b3,stroke:#000;
classDef hook-first fill:#fff2b3,stroke:#000;
Expand All @@ -9,25 +9,25 @@ flowchart TB
augmentchunkhash("augmentChunkHash"):::hook-sequential-sync
click augmentchunkhash "/guide/en/#augmentchunkhash" _parent

banner("banner"):::hook-parallel
banner("banner"):::hook-sequential
click banner "/guide/en/#banner" _parent

closebundle("closeBundle"):::hook-parallel
click closebundle "/guide/en/#closebundle" _parent

footer("footer"):::hook-parallel
footer("footer"):::hook-sequential
click footer "/guide/en/#footer" _parent

generatebundle("generateBundle"):::hook-sequential
click generatebundle "/guide/en/#generatebundle" _parent

intro("intro"):::hook-parallel
intro("intro"):::hook-sequential
click intro "/guide/en/#intro" _parent

outputoptions("outputOptions"):::hook-sequential-sync
click outputoptions "/guide/en/#outputoptions" _parent

outro("outro"):::hook-parallel
outro("outro"):::hook-sequential
click outro "/guide/en/#outro" _parent

renderchunk("renderChunk"):::hook-sequential
Expand All @@ -54,27 +54,47 @@ flowchart TB

outputoptions
--> renderstart
--> banner & footer & intro & outro
--> beforerenderdynamicimport(( ))
--> beforeaugmentchunkhash(( ))
--> |each chunk|augmentchunkhash
--> renderchunk
.-> generatebundle
--> writebundle
.-> closebundle
--> |each chunk|beforerenderdynamicimport

beforerenderdynamicimport
--> |"each import()"|renderdynamicimport
--> beforeaugmentchunkhash
afteraddons
--> |each chunk|renderchunk

augmentchunkhash
--> |each import.meta.*|beforeimportmeta(( ))
--> |import.meta.url|resolvefileurl
.-> renderchunk
--> generatebundle
--> writebundle
.-> closebundle

beforeimportmeta
--> |other|resolveimportmeta
.-> renderchunk
subgraph generateChunks [" "]
direction TB
beforerenderdynamicimport(( ))
---> beforeresolveimportmeta(( ))
----> beforereaddons(( ))
--> banner & footer & intro & outro
--> afteraddons(( ))
.-> |next chunk|beforerenderdynamicimport

beforerenderdynamicimport
--> |"each import()"|renderdynamicimport
--> beforerenderdynamicimport

beforeresolveimportmeta
--> |each import.meta.*|beforeimportmeta(( ))
--> |import.meta.url|resolvefileurl
--> afterresolveimportmeta(( ))

beforeimportmeta
--> |other|resolveimportmeta
--> afterresolveimportmeta

afterresolveimportmeta
--> beforeresolveimportmeta
end

renderchunk
--> augmentchunkhash
.-> |next chunk|renderchunk

style generateChunks stroke-width:0px;

rendererror
.-> closebundle