From 5e8385a86390eacf0d5cdc3cc785326da467d7d2 Mon Sep 17 00:00:00 2001 From: Lukas Taegert-Atkinson Date: Mon, 15 Aug 2022 06:37:54 +0200 Subject: [PATCH] [v3.0] Better sourcemap emission (#4605) * [v3.0] Deprecate Node 12 (#4548) * [v3.0] Remove actively deprecated features, show warnings for other deprecated features (#4552) * Remove all active deprecations * Make all inactive deprecations active * Try to make test more stable * Update CLI help screen * [v3.0] New hashing algorithm that "fixes (nearly) everything" (#4543) * Initial new hashing idea * Simplify external import path generation 197 broken tests left * Use correct file names in chunk info 197 broken tests left * Implement first draft for hashing algorithm 189 broken tests left * Remove active deprecations this.emitAsset this.emitChunk this.getAssetFileName this.getChunkFileName import.meta.ROLLUP_ASSET_URL_ import.meta.ROLLUP_CHUNK_URL_ * Reduce render parameters * Always scan all chunks for hashes * Fix asset emission and remaining tests * Reintroduce augmentChunkHash and get OutputChunk by converting RenderedChunk * Provide chunk graph in renderChunk * Handle hash collisions * Remove deprecated hacky asset emission * Allow to configure hash sizes per file * Update documentation * Extend tests * Minor improvements * Improve documentation about hashing * Replace hash in sourcemap file * Provide ChunkInfo in banner/footer/intro/outro * Extract hashing logic * Clean up hashing logic * Add ExternalChunk wrapper * Store inputBase on Chunk * Store snippets on Chunk * Align chunk interfaces * Reduce this. property access * Move dynamicImportFunction warning to options normalization * Restructure rendering logic * Do not run on Node 10 * Update documentation * Try to fix Windows tests * Improve coverage * Remove graph background colors 3.0.0-0 * [v3.0] Run output plugins last (#3846) * fix: run output plugins last * Add test Co-authored-by: Lukas Taegert-Atkinson * [v3.0] Convert build scripts to ESM, update dependencies (#4558) * Convert scripts to ESM, update dependencies * Fix lint issue 3.0.0-1 * [v3.0] Better esm config file support (#4574) * More precise native ESM support check * Use import to load .js config file if package type is module * Update and add tests handling type module * Remove Node version check and simplify logic * Update documentation * Document how to replace __dirname and import JSON Co-authored-by: Linus Miller * [v3.0] Rework file name patterns when preserving modules (#4565) * Put preserveModules path into Chunk name * Use regular entryFileNames logic * Clarify documentation for `preserveModules` * Improve coverage * Improve wording in docs * [v3.0] Show deprecation warning for maxParallelFileReads (#4575) * [v3.0] Restructure timings (#4566) * [v3.0] Change default for makeAbsoluteExternalsRelative (#4567) * [v3.0] Change default for output.generatedCode.reservedNamesAsProps (#4568) * [v3.0] Change default for preserveEntrySignatures to exports-only (#4576) * Port doc changes from #4572 and #4583 to 3.0 (#4592) * [v3.0] Refine errors and warnings (#4579) * [v3.0] New hashing algorithm that "fixes (nearly) everything" (#4543) * Initial new hashing idea * Simplify external import path generation 197 broken tests left * Use correct file names in chunk info 197 broken tests left * Implement first draft for hashing algorithm 189 broken tests left * Remove active deprecations this.emitAsset this.emitChunk this.getAssetFileName this.getChunkFileName import.meta.ROLLUP_ASSET_URL_ import.meta.ROLLUP_CHUNK_URL_ * Reduce render parameters * Always scan all chunks for hashes * Fix asset emission and remaining tests * Reintroduce augmentChunkHash and get OutputChunk by converting RenderedChunk * Provide chunk graph in renderChunk * Handle hash collisions * Remove deprecated hacky asset emission * Allow to configure hash sizes per file * Update documentation * Extend tests * Minor improvements * Improve documentation about hashing * Replace hash in sourcemap file * Provide ChunkInfo in banner/footer/intro/outro * Extract hashing logic * Clean up hashing logic * Add ExternalChunk wrapper * Store inputBase on Chunk * Store snippets on Chunk * Align chunk interfaces * Reduce this. property access * Move dynamicImportFunction warning to options normalization * Restructure rendering logic * Do not run on Node 10 * Update documentation * Try to fix Windows tests * Improve coverage * Remove graph background colors 3.0.0-0 * [v3.0] Change default for output.generatedCode.reservedNamesAsProps (#4568) * [v3.0] New hashing algorithm that "fixes (nearly) everything" (#4543) * Initial new hashing idea * Simplify external import path generation 197 broken tests left * Use correct file names in chunk info 197 broken tests left * Implement first draft for hashing algorithm 189 broken tests left * Remove active deprecations this.emitAsset this.emitChunk this.getAssetFileName this.getChunkFileName import.meta.ROLLUP_ASSET_URL_ import.meta.ROLLUP_CHUNK_URL_ * Reduce render parameters * Always scan all chunks for hashes * Fix asset emission and remaining tests * Reintroduce augmentChunkHash and get OutputChunk by converting RenderedChunk * Provide chunk graph in renderChunk * Handle hash collisions * Remove deprecated hacky asset emission * Allow to configure hash sizes per file * Update documentation * Extend tests * Minor improvements * Improve documentation about hashing * Replace hash in sourcemap file * Provide ChunkInfo in banner/footer/intro/outro * Extract hashing logic * Clean up hashing logic * Add ExternalChunk wrapper * Store inputBase on Chunk * Store snippets on Chunk * Align chunk interfaces * Reduce this. property access * Move dynamicImportFunction warning to options normalization * Restructure rendering logic * Do not run on Node 10 * Update documentation * Try to fix Windows tests * Improve coverage * Remove graph background colors 3.0.0-0 * Rework warnings and errors * Refine some error messages * Reduce number of different props of errors * All errors are declared in error.ts * Use name RollupError for errors that do not have a cause * Extend documentation * [v3.0] Browser build (#4593) * [v3.0] New hashing algorithm that "fixes (nearly) everything" (#4543) * Initial new hashing idea * Simplify external import path generation 197 broken tests left * Use correct file names in chunk info 197 broken tests left * Implement first draft for hashing algorithm 189 broken tests left * Remove active deprecations this.emitAsset this.emitChunk this.getAssetFileName this.getChunkFileName import.meta.ROLLUP_ASSET_URL_ import.meta.ROLLUP_CHUNK_URL_ * Reduce render parameters * Always scan all chunks for hashes * Fix asset emission and remaining tests * Reintroduce augmentChunkHash and get OutputChunk by converting RenderedChunk * Provide chunk graph in renderChunk * Handle hash collisions * Remove deprecated hacky asset emission * Allow to configure hash sizes per file * Update documentation * Extend tests * Minor improvements * Improve documentation about hashing * Replace hash in sourcemap file * Provide ChunkInfo in banner/footer/intro/outro * Extract hashing logic * Clean up hashing logic * Add ExternalChunk wrapper * Store inputBase on Chunk * Store snippets on Chunk * Align chunk interfaces * Reduce this. property access * Move dynamicImportFunction warning to options normalization * Restructure rendering logic * Do not run on Node 10 * Update documentation * Try to fix Windows tests * Improve coverage * Remove graph background colors 3.0.0-0 * Set up browser build * Generate commithash inline * Remove execa * Chmod executable from rollup config * Adjust pkg.files * Copy types into build * Work on release script * Continue work on release script * Continue work on release script * Finish release script * Push correct REPL artefacts * Add comments to released PRs and issues * fixup! [v3.0] New hashing algorithm that "fixes (nearly) everything" (#4543) * Add Rollup-in-browser example * [v3.0] Use named export for loadConfigFile (#4581) * [v3.0] Use named export for loadConfigFile * Only expose files in dist with their full names * Move browser sources to src subfolder * 3.0.0-3 * Fix release script * [v3.0] Use "node:" prefix for imports of node builtins (#4596) * Support inline sourcemaps * Emit sourcemaps as assets and add comments before generateBundle Co-authored-by: Alec Larson <1925840+aleclarson@users.noreply.github.com> Co-authored-by: Linus Miller Co-authored-by: Bertrand Guay-Paquet 3.0.0-4 --- browser/package.json | 2 +- cli/run/build.ts | 13 +------ package.json | 2 +- src/rollup/rollup.ts | 29 +------------- src/utils/renderChunks.ts | 38 ++++++++++++++++--- .../samples/stdout-single-input/_expected.js | 1 - .../_config.js | 26 +++++++++++++ .../sourcemap-inline-generatebundle/main.js | 1 + .../sourcemap-true-generatebundle/_config.js | 33 ++++++++++++++++ .../sourcemap-true-generatebundle/main.js | 1 + .../_config.js | 2 +- .../_config.js | 2 +- 12 files changed, 101 insertions(+), 49 deletions(-) create mode 100644 test/function/samples/sourcemap-inline-generatebundle/_config.js create mode 100644 test/function/samples/sourcemap-inline-generatebundle/main.js create mode 100644 test/function/samples/sourcemap-true-generatebundle/_config.js create mode 100644 test/function/samples/sourcemap-true-generatebundle/main.js diff --git a/browser/package.json b/browser/package.json index e5bc3c891b4..c566f478e7f 100644 --- a/browser/package.json +++ b/browser/package.json @@ -1,6 +1,6 @@ { "name": "@rollup/browser", - "version": "3.0.0-3", + "version": "3.0.0-4", "description": "Next-generation ES module bundler browser build", "main": "dist/rollup.browser.js", "module": "dist/es/rollup.browser.js", diff --git a/cli/run/build.ts b/cli/run/build.ts index b2c9418be27..9c8fb8784c1 100644 --- a/cli/run/build.ts +++ b/cli/run/build.ts @@ -5,7 +5,6 @@ import type { MergedRollupOptions } from '../../src/rollup/types'; import { bold, cyan, green } from '../../src/utils/colors'; import { errOnlyInlineSourcemapsForStdout } from '../../src/utils/error'; import relativeId from '../../src/utils/relativeId'; -import { SOURCEMAPPING_URL } from '../../src/utils/sourceMappingURL'; import { handleError, stderr } from '../logging'; import type { BatchWarnings } from './batchWarnings'; import { printTimings } from './timings'; @@ -37,20 +36,10 @@ export default async function build( if (output.sourcemap && output.sourcemap !== 'inline') { handleError(errOnlyInlineSourcemapsForStdout()); } - const { output: outputs } = await bundle.generate(output); for (const file of outputs) { - let source: string | Uint8Array; - if (file.type === 'asset') { - source = file.source; - } else { - source = file.code; - if (output.sourcemap === 'inline') { - source += `\n//# ${SOURCEMAPPING_URL}=${file.map!.toUrl()}\n`; - } - } if (outputs.length > 1) process.stdout.write(`\n${cyan(bold(`//→ ${file.fileName}:`))}\n`); - process.stdout.write(source as Buffer); + process.stdout.write(file.type === 'asset' ? file.source : file.code); } if (!silent) { warnings.flush(); diff --git a/package.json b/package.json index 800dab8a17c..70e6f3946c7 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "rollup", - "version": "3.0.0-3", + "version": "3.0.0-4", "description": "Next-generation ES module bundler", "main": "dist/rollup.js", "module": "dist/es/rollup.js", diff --git a/src/rollup/rollup.ts b/src/rollup/rollup.ts index d039929464b..b0a8f441cd3 100644 --- a/src/rollup/rollup.ts +++ b/src/rollup/rollup.ts @@ -15,9 +15,8 @@ import { catchUnfinishedHookActions } from '../utils/hookActions'; import { normalizeInputOptions } from '../utils/options/normalizeInputOptions'; import { normalizeOutputOptions } from '../utils/options/normalizeOutputOptions'; import type { GenericConfigObject } from '../utils/options/options'; -import { basename, dirname, resolve } from '../utils/path'; +import { dirname, resolve } from '../utils/path'; import { ANONYMOUS_OUTPUT_PLUGIN_PREFIX, ANONYMOUS_PLUGIN_PREFIX } from '../utils/pluginUtils'; -import { SOURCEMAPPING_URL } from '../utils/sourceMappingURL'; import { getTimings, initialiseTimers, timeEnd, timeStart } from '../utils/timers'; import type { NormalizedInputOptions, @@ -282,31 +281,7 @@ async function writeOutputFile( // 'recursive: true' does not throw if the folder structure, or parts of it, already exist await fs.mkdir(dirname(fileName), { recursive: true }); - let writeSourceMapPromise: Promise | undefined; - let source: string | Uint8Array; - if (outputFile.type === 'asset') { - source = outputFile.source; - } else { - source = outputFile.code; - if (outputOptions.sourcemap && outputFile.map) { - let url: string; - if (outputOptions.sourcemap === 'inline') { - url = outputFile.map.toUrl(); - } else { - const { sourcemapBaseUrl } = outputOptions; - const sourcemapFileName = `${basename(outputFile.fileName)}.map`; - url = sourcemapBaseUrl - ? new URL(sourcemapFileName, sourcemapBaseUrl).toString() - : sourcemapFileName; - writeSourceMapPromise = fs.writeFile(`${fileName}.map`, outputFile.map.toString()); - } - if (outputOptions.sourcemap !== 'hidden') { - source += `//# ${SOURCEMAPPING_URL}=${url}\n`; - } - } - } - - return Promise.all([fs.writeFile(fileName, source), writeSourceMapPromise]); + return fs.writeFile(fileName, outputFile.type === 'asset' ? outputFile.source : outputFile.code); } /** diff --git a/src/utils/renderChunks.ts b/src/utils/renderChunks.ts index ac9ee1c6d4a..00ed35df629 100644 --- a/src/utils/renderChunks.ts +++ b/src/utils/renderChunks.ts @@ -19,7 +19,8 @@ import { replacePlaceholdersWithDefaultAndGetContainedPlaceholders, replaceSinglePlaceholder } from './hashPlaceholders'; -import { normalize, resolve } from './path'; +import { basename, normalize, resolve } from './path'; +import { SOURCEMAPPING_URL } from './sourceMappingURL'; import { timeEnd, timeStart } from './timers'; interface HashResult { @@ -70,7 +71,9 @@ export async function renderChunks( renderedChunksByPlaceholder, hashesByPlaceholder, outputBundle, - nonHashedChunksWithPlaceholders + nonHashedChunksWithPlaceholders, + pluginDriver, + outputOptions ); timeEnd('transform chunks', 2); @@ -286,20 +289,45 @@ function addChunksToBundle( renderedChunksByPlaceholder: Map, hashesByPlaceholder: Map, outputBundle: OutputBundleWithPlaceholders, - nonHashedChunksWithPlaceholders: RenderedChunkWithPlaceholders[] + nonHashedChunksWithPlaceholders: RenderedChunkWithPlaceholders[], + pluginDriver: PluginDriver, + options: NormalizedOutputOptions ) { for (const { chunk, code, fileName, map } of renderedChunksByPlaceholder.values()) { - const updatedCode = replacePlaceholders(code, hashesByPlaceholder); + let updatedCode = replacePlaceholders(code, hashesByPlaceholder); const finalFileName = replacePlaceholders(fileName, hashesByPlaceholder); if (map) { map.file = replacePlaceholders(map.file, hashesByPlaceholder); + updatedCode += emitSourceMapAndGetComment(finalFileName, map, pluginDriver, options); } outputBundle[finalFileName] = chunk.generateOutputChunk(updatedCode, map, hashesByPlaceholder); } for (const { chunk, code, fileName, map } of nonHashedChunksWithPlaceholders) { - const updatedCode = hashesByPlaceholder.size + let updatedCode = hashesByPlaceholder.size ? replacePlaceholders(code, hashesByPlaceholder) : code; + if (map) { + updatedCode += emitSourceMapAndGetComment(fileName, map, pluginDriver, options); + } outputBundle[fileName] = chunk.generateOutputChunk(updatedCode, map, hashesByPlaceholder); } } + +function emitSourceMapAndGetComment( + fileName: string, + map: SourceMap, + pluginDriver: PluginDriver, + { sourcemap, sourcemapBaseUrl }: NormalizedOutputOptions +) { + let url: string; + if (sourcemap === 'inline') { + url = map.toUrl(); + } else { + const sourcemapFileName = `${basename(fileName)}.map`; + url = sourcemapBaseUrl + ? new URL(sourcemapFileName, sourcemapBaseUrl).toString() + : sourcemapFileName; + pluginDriver.emitFile({ fileName: `${fileName}.map`, source: map.toString(), type: 'asset' }); + } + return sourcemap === 'hidden' ? '' : `//# ${SOURCEMAPPING_URL}=${url}\n`; +} diff --git a/test/cli/samples/stdout-single-input/_expected.js b/test/cli/samples/stdout-single-input/_expected.js index 70449540ecd..c7ae3534c56 100644 --- a/test/cli/samples/stdout-single-input/_expected.js +++ b/test/cli/samples/stdout-single-input/_expected.js @@ -1,3 +1,2 @@ console.log( 42 ); - //# sourceMappingURL=data:application/json;charset=utf-8;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoibWFpbi5qcyIsInNvdXJjZXMiOlsibWFpbi5qcyJdLCJzb3VyY2VzQ29udGVudCI6WyJjb25zb2xlLmxvZyggNDIgKTtcbiJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiQUFBQSxPQUFPLENBQUMsR0FBRyxFQUFFLEVBQUUsRUFBRSJ9 diff --git a/test/function/samples/sourcemap-inline-generatebundle/_config.js b/test/function/samples/sourcemap-inline-generatebundle/_config.js new file mode 100644 index 00000000000..85720d5db90 --- /dev/null +++ b/test/function/samples/sourcemap-inline-generatebundle/_config.js @@ -0,0 +1,26 @@ +const assert = require('assert'); + +module.exports = { + description: 'includes inline sourcemap comments in generateBundle hook', + options: { + plugins: [ + { + name: 'test', + generateBundle(options, bundle) { + assert.deepStrictEqual(Object.keys(bundle), ['main.js']); + assert.strictEqual( + bundle['main.js'].code, + `'use strict'; + +var main = 42; + +module.exports = main; +//# sourceMappingURL=data:application/json;charset=utf-8;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoibWFpbi5qcyIsInNvdXJjZXMiOlsibWFpbi5qcyJdLCJzb3VyY2VzQ29udGVudCI6WyJleHBvcnQgZGVmYXVsdCA0MjtcbiJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiOztBQUFBLFdBQWUsRUFBRTs7OzsifQ== +` + ); + } + } + ], + output: { sourcemap: 'inline' } + } +}; diff --git a/test/function/samples/sourcemap-inline-generatebundle/main.js b/test/function/samples/sourcemap-inline-generatebundle/main.js new file mode 100644 index 00000000000..7a4e8a723a4 --- /dev/null +++ b/test/function/samples/sourcemap-inline-generatebundle/main.js @@ -0,0 +1 @@ +export default 42; diff --git a/test/function/samples/sourcemap-true-generatebundle/_config.js b/test/function/samples/sourcemap-true-generatebundle/_config.js new file mode 100644 index 00000000000..74edbb0d46a --- /dev/null +++ b/test/function/samples/sourcemap-true-generatebundle/_config.js @@ -0,0 +1,33 @@ +const assert = require('assert'); + +module.exports = { + description: 'emits sourcemaps before generateBundle hook', + options: { + plugins: [ + { + name: 'test', + generateBundle(options, bundle) { + assert.deepStrictEqual(Object.keys(bundle), ['main.js', 'main.js.map']); + assert.strictEqual( + bundle['main.js'].code, + `'use strict'; + +var main = 42; + +module.exports = main; +//# sourceMappingURL=main.js.map +` + ); + assert.deepStrictEqual(bundle['main.js.map'], { + fileName: 'main.js.map', + name: undefined, + source: + '{"version":3,"file":"main.js","sources":["main.js"],"sourcesContent":["export default 42;\\n"],"names":[],"mappings":";;AAAA,WAAe,EAAE;;;;"}', + type: 'asset' + }); + } + } + ], + output: { sourcemap: true } + } +}; diff --git a/test/function/samples/sourcemap-true-generatebundle/main.js b/test/function/samples/sourcemap-true-generatebundle/main.js new file mode 100644 index 00000000000..7a4e8a723a4 --- /dev/null +++ b/test/function/samples/sourcemap-true-generatebundle/main.js @@ -0,0 +1 @@ +export default 42; diff --git a/test/sourcemaps/samples/existing-external-sourcemaps-combined/_config.js b/test/sourcemaps/samples/existing-external-sourcemaps-combined/_config.js index 73a37f7cc12..3497a5ae1dc 100644 --- a/test/sourcemaps/samples/existing-external-sourcemaps-combined/_config.js +++ b/test/sourcemaps/samples/existing-external-sourcemaps-combined/_config.js @@ -3,6 +3,6 @@ const assert = require('assert'); module.exports = { description: 'removes sourcemap comments', async test(code) { - assert.ok(!code.includes('sourceMappingURL')); + assert.ok(!code.includes('sourceMappingURL=main.js.map')); } }; diff --git a/test/sourcemaps/samples/existing-inline-sourcemaps-combined/_config.js b/test/sourcemaps/samples/existing-inline-sourcemaps-combined/_config.js index 36c14ef2529..62926cc356a 100644 --- a/test/sourcemaps/samples/existing-inline-sourcemaps-combined/_config.js +++ b/test/sourcemaps/samples/existing-inline-sourcemaps-combined/_config.js @@ -3,6 +3,6 @@ const assert = require('assert'); module.exports = { description: 'removes existing inline sourcemaps', async test(code) { - assert.ok(!code.includes('sourceMappingURL')); + assert.ok(!code.includes('sourceMappingURL=data')); } };