🐛 bug report: parcel does not load existing sourcemaps of pre-transformed JS assets #741

lxe · 2018-02-04T22:52:18Z

This is a 🐛 bug report

General summary:

Importing/requiring a pre-built file does not load its sourcemaps.

For example, mapbox-gl-js exports a pre-packaged main which lives in node_modules/mapbox-gl/dist/mapbox-gl.js. It also contains a sourceMappingUrl //# sourceMappingURL=mapbox-gl.js.map. There is a mapbox-gl.js.map in dist/ as well.

When doing import mapboxgl from 'mapbox-gl' , parcel correctly bundles node_modules/mapbox-gl/dist/mapbox-gl.js, but does not load and re-process its sourcemap.

I haven't gotten far with debugging it, but it seems that that file does not satisfy any of the conditions that would mark it for transformation (https://github.com/parcel-bundler/parcel/blob/master/src/transforms/babel.js#L29).

Artificially marking it for a transform (by hardcoding something like if (/mapbox-gl\.js/.test(asset.name)) return true will run the babel compiler on it, which is also not what we want.

To reproduce:

yarn add mapbox-gl

Inspect the entrypoint:

❯ jq .main node_modules/mapbox-gl/package.json
"dist/mapbox-gl.js"

Inspect the presence of the entrypoint and the map:

❯ ls node_modules/mapbox-gl/dist/
mapbox-gl-dev.js mapbox-gl.css    mapbox-gl.js     mapbox-gl.js.map svg

Looks like we're pointing to the sourcemap correctly:

❯ ag "sourceMapping"  node_modules/mapbox-gl/dist/mapbox-gl.js
566://# sourceMappingURL=mapbox-gl.js.map

Import the file, and package it with parcel:

// index.js
import mapbox from 'mapbox-gl`

// index.html
<html><body><script src="./index.js"></script></body></html>

❯ parcel index.html
Server running at http://localhost:64815
✨  Built in 1.88s.

Verify that the sourcemap of mapbox-gl.js does not get repackaged:

🌍 Your Environment

Software	Version(s)
Parcel	1.5.1
Node	v8.9.3
npm/Yarn	1.3.2
Operating System	Darwin

The text was updated successfully, but these errors were encountered:

lxe · 2018-02-04T23:27:43Z

Basically we need something like https://github.com/webpack-contrib/source-map-loader

Mickyfen17 · 2018-02-25T03:09:57Z

I've just come across this same issue today when trying to use mapbox-gl & parcel together.

devongovett · 2018-02-27T15:45:03Z

This seems easy enough to do. We already have the infrastructure in place, just need to parse the existing source map url comment in the code and load it.

Offirmo · 2018-05-31T08:06:18Z

Tried parcel, was delighted, then put a "debugger" statement...

Please fix this!

snstrauss · 2018-06-19T07:33:58Z

I agree.
Using a 'debugger;' statement causes the debug session to pause at random places across the code.

What can I do about it?

lxe · 2018-06-20T18:17:08Z

What does debugger; have to do with this issue? @Offirmo @snstrauss Did you mean to comment elsewhere?

Offirmo · 2018-06-21T03:06:18Z

@lxe think about it...

[edit] using "debugger" make the source map issue apparent by showing code mismatched to where the debugger clause is.

iFwu · 2019-01-11T06:09:58Z

any updates？

medmin · 2020-07-09T01:35:47Z

any updates ?

Bug-Reaper · 2020-09-29T07:11:05Z

Any insight into how to debug issues where parcel fails to load existing sourcemaps. Similar problem with this npm module: https://github.com/mfbx9da4/deep-email-validator

lemraus · 2020-10-12T13:35:22Z

I am willing to re-open this issue as I have come across this problem earlier today, and after taking a look at #1349 I understood where the problem is. #1349 did not entirely solve this issue, below is an explanation as to why.

From the source code of #1349, in file src/assets/JSAsset.js, line 26:

const SOURCEMAP_RE = /\/\/\s*[@#]\s*sourceMappingURL\s*=\s*([^\s]+)/;

Then later, line 114:

let match = this.contents.match(SOURCEMAP_RE);

This would work fine if all sourceMappingURL statements started with //.
I have recently started to work with neovis.js, a client visualization tool for Neo4j databases. Now take a look at their minified neovis.js file, line 95, column 21247:

o&&btoa&&(i+="\n/*# sourceMappingURL=data:application/json;base64,".concat(btoa(unescape(encodeURIComponent(JSON.stringify(o))))," */"))

Uh-oh, looks like folks at neovis.js felt like using multiline comments (/* */) instead of the usual // for sourceMappingURL.
I started thinking that it might be a neovis.js related issue, however it turns out things work fine with webpack.

After looking at the source code of parcel v2, it seems that the regexp has been corrected since v1. In file packages/core/utils/src/sourcemap.js, line 7:

export const SOURCEMAP_RE: RegExp = /(?:\/\*|\/\/)\s*[@#]\s*sourceMappingURL\s*=\s*([^\s*]+)(?:\s*\*\/)?\s*$/;

It remains a problem as installing parcel with a simple yarn add parcel still defaults to v1.
For now a temporary fix would be to upgrade to v2, but using the @next version of a package is not an acceptable fix, IMHO.

I think this could be solved by simply copying the regexp from v2 to the v1 source code, should I file a pull request with the changes?

jzohrab · 2021-04-06T00:15:14Z

Hi all, I'm running into this now (parcel version 1.12.5), with a js library called Vexflow. It has the same commenting style noted above:

o&&btoa&&(n+="\n/*# sourceMappingURL=data:application/json;base64,".concat(btoa(unescape(encodeURIComponent(JSON.stringify(o))))," */"))

@lemraus I'm happy to create and submit this PR with your suggested regex above, but I don't see an obvious v1 branch or tag to work off of. @devongovett - any suggestions? Thanks!

edit: actually, it might be trickier than I expected. My node_modules/parcel-bundler/src/utils/loadSourceMap.js already had const SOURCEMAP_RE = /(?:\/\*|\/\/)\s*[@#]\s*sourceMappingURL\s*=\s*([^\s*]+)(?:\s*\*\/)?/;, so I added the error details to the try/catch:

    } catch (e) {
      logger.warn(
        `Could not load existing sourcemap of "${asset.relativeName}, error = ${e}".`
      );

and got Could not load existing sourcemap of "../node_modules/vextab/dist/div.prod.js, error = SyntaxError: Unexpected token r in JSON at position 0".

Earlier in the code, it's doing

      if (dataURLMatch) {
        filename = asset.name;
        logger.warn(`a. making buffer from ${dataURLMatch[1]}`)   // <-- added
        json = Buffer.from(dataURLMatch[1], 'base64').toString();

which is outputting

a. making buffer from ".concat(btoa(unescape(encodeURIComponent(JSON.stringify(o)))),"

So the regex may be off, at least for this particular case.

lxe changed the title ~~🐛 bug report: parcel does not load sourcemaps for JS assets not marked for transformation.~~ 🐛 bug report: parcel does not load sourcemaps of pre-transformed JS assets not marked for transformation. Feb 4, 2018

lxe changed the title ~~🐛 bug report: parcel does not load sourcemaps of pre-transformed JS assets not marked for transformation.~~ 🐛 bug report: parcel does not load existing sourcemaps of pre-transformed JS assets Feb 4, 2018

devongovett added 🙋‍♀️ Feature 🙏 Help Wanted labels Feb 27, 2018

devongovett added this to the v1.8.0 milestone Apr 1, 2018

DeMoorJasper added the Good First Issue label Apr 18, 2018

devongovett modified the milestones: v1.8.0, v1.9.0 May 1, 2018

DeMoorJasper mentioned this issue May 12, 2018

Import existing sourcemaps from libraries #1349

Merged

devongovett closed this as completed in #1349 Jul 22, 2018

mbj36 mentioned this issue Apr 1, 2019

Could not load existing sourcemap mbj36/vue-burger-menu#51

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 bug report: parcel does not load existing sourcemaps of pre-transformed JS assets #741

🐛 bug report: parcel does not load existing sourcemaps of pre-transformed JS assets #741

lxe commented Feb 4, 2018 •

edited

lxe commented Feb 4, 2018

Mickyfen17 commented Feb 25, 2018 •

edited

devongovett commented Feb 27, 2018

Offirmo commented May 31, 2018

snstrauss commented Jun 19, 2018

lxe commented Jun 20, 2018

Offirmo commented Jun 21, 2018 •

edited

iFwu commented Jan 11, 2019

medmin commented Jul 9, 2020

Bug-Reaper commented Sep 29, 2020

lemraus commented Oct 12, 2020 •

edited

jzohrab commented Apr 6, 2021 •

edited

🐛 bug report: parcel does not load existing sourcemaps of pre-transformed JS assets #741

🐛 bug report: parcel does not load existing sourcemaps of pre-transformed JS assets #741

Comments

lxe commented Feb 4, 2018 • edited

🌍 Your Environment

lxe commented Feb 4, 2018

Mickyfen17 commented Feb 25, 2018 • edited

devongovett commented Feb 27, 2018

Offirmo commented May 31, 2018

snstrauss commented Jun 19, 2018

lxe commented Jun 20, 2018

Offirmo commented Jun 21, 2018 • edited

iFwu commented Jan 11, 2019

medmin commented Jul 9, 2020

Bug-Reaper commented Sep 29, 2020

lemraus commented Oct 12, 2020 • edited

jzohrab commented Apr 6, 2021 • edited

lxe commented Feb 4, 2018 •

edited

Mickyfen17 commented Feb 25, 2018 •

edited

Offirmo commented Jun 21, 2018 •

edited

lemraus commented Oct 12, 2020 •

edited

jzohrab commented Apr 6, 2021 •

edited