use rollup to bundle runmode-standalone and runmode-node

[BrianHung]

This PR changes the runmode addon to use Rollup to generate runmode-standalone and runmode-node, keeping methods always up to date with runmode.js, src/modes, src/utils/misc.js and /src/utils/StringStream.js. This solves #6311 and #6313 with a more programmatic approach. To generate the two files, we run npm run build since I modified the main rollup.config.js.

Two notes:

• I've only 'tested' the runmode-standalone with XML highlighting. I don't see any glaring errors which would make this not backwards-compatible with previous versions (besides the different names, but that can be changed easily in the rollup.config.js file)
• One thing that comes with using library addon/runmode file to generate the runMode methods is that the following is included in the two files. We could get rid of this by creating a separate file that 'mirrors' the runMode method and which doesn't have this header. I wouldn't think this header causes issues though, since this is included in every language in mode/[mode]/[mode].js.

(function(mod) {
  if (typeof exports == "object" && typeof module == "object") // CommonJS
    mod(require("../../lib/codemirror"));
  else if (typeof define == "function" && define.amd) // AMD
    define(["../../lib/codemirror"], mod);
  else // Plain browser env
    mod(CodeMirror);
})

// to remove header, create a file with _just_ the runMode method with no header 
// and change the rollup.config.js to use that as an input
    input: ["addon/runmode/src/runmode-browser.js", "addon/runmode/src/justRunMode.js"] 

[marijnh]

Thanks for working on this. There's some issues, though...

• Changing the name of runmode.node.js to runmode-node.js is a breaking change, and doesn't seem necessary
• The runmode-node.js from this PR doesn't work.
• Build artifacts shouldn't be checked into the repository—like lib/codemirror.js, they'll get created by the build process before publishing
• We support browsers that don't have Object.assign yet.
• It would probably be easier to factor the runMode function into an ES6 file and import it from three different wrapper files to create the three versions of the addon
• Either the liter needs to be configured to treat the content of addon/runmode/src as ES6, or those files could be moved to src/addon/runode

Do you want to continue to work on this or would you prefer if I just merge your two other PRs and we leave these duplicated?

[BrianHung]

You can merge the other two PRs for now since I don't know how long it'll take to resolve the issues brought up; I can continue to look into using rollup to generate the runmode-standalone and runmode.node.

We support browsers that don't have Object.assign yet.

Ah, I think we can use the copyObj from src/utils/misc.js for this. It's pretty close to the suggested polyfill except without supporting varargs.

It would probably be easier to factor the runMode function into an ES6 file and import it from three different wrapper files to create the three versions of the addon

I had this in mind, since I saw that the src/modes.js was modularized as a ES6 file and is easy to use and import. I'll move to ES6 since it seems like a better approach than what I'm doing currently.

Either the liter needs to be configured to treat the content of addon/runmode/src as ES6, or those files could be moved to src/addon/runode

Moving it to src/addon/runmode sounds good.

[BrianHung]

• Changing the name of runmode.node.js to runmode-node.js is a breaking change, and doesn't seem necessary

Reverted the change within src/ and rollup.config.js.

• The runmode-node.js from this PR doesn't work.

I believe this is fixed by taking all the methods attached to CodeMirror and instead attaching them onto exports. I used the runmode.node.js with the example from the runmode demo and it worked.

runmode-node demo

const CodeMirror = require("./addon/runmode/runmode.node.js");
require("./mode/xml/xml");

// Log CodeMirror instance to visually check methods.
console.log(CodeMirror);

// Example code from mode runner demo.
var src = `
<foobar>
  <blah>Enter your xml here and press the button below to display it as highlighted by the CodeMirror XML mode</blah>
  <tag2 foo="2" bar="&quot;bar&quot;"/>
</foobar>
`

// Node.js doesn't have dom so we 
var pre = `<pre class="cm-s-default">\n`;

// Call runMode method on example code.
CodeMirror.runMode(src, "xml", (token, tokenClass) => {
  pre = `${pre}<span class="cm-${tokenClass}"\>${token}</span>\n`
});

pre = `${pre}</pre>\n`;

// Log the syntax highlighted xml code.
console.log(pre);

runmode demo logs

{
  StringStream: [Function: StringStream],
  modes: { null: [Function (anonymous)], xml: [Function (anonymous)] },
  mimeModes: {
    'text/plain': 'null',
    'text/xml': 'xml',
    'application/xml': 'xml',
    'text/html': { name: 'xml', htmlMode: true }
  },
  defineMode: [Function: defineMode],
  defineMIME: [Function: defineMIME],
  resolveMode: [Function: resolveMode],
  getMode: [Function: getMode],
  modeExtensions: {},
  extendMode: [Function: extendMode],
  copyState: [Function: copyState],
  innerMode: [Function: innerMode],
  startState: [Function: startState],
  registerGlobalHelper: [Function: min],
  registerHelper: [Function: min],
  splitLines: [Function (anonymous)],
  defaults: { indentUnit: 2 },
  runMode: [Function (anonymous)]
}
<pre class="cm-s-default">
<span class="cm-undefined">
</span>
<span class="cm-tag bracket"><</span>
<span class="cm-tag">foobar</span>
<span class="cm-tag bracket">></span>
<span class="cm-undefined">
</span>
<span class="cm-null">  </span>
<span class="cm-tag bracket"><</span>
<span class="cm-tag">blah</span>
<span class="cm-tag bracket">></span>
<span class="cm-null">Enter your xml here and press the button below to display it as highlighted by the CodeMirror XML mode</span>
<span class="cm-tag bracket"></</span>
<span class="cm-tag">blah</span>
<span class="cm-tag bracket">></span>
<span class="cm-undefined">
</span>
<span class="cm-null">  </span>
<span class="cm-tag bracket"><</span>
<span class="cm-tag">tag2</span>
<span class="cm-null"> </span>
<span class="cm-attribute">foo</span>
<span class="cm-null">=</span>
<span class="cm-string">"2"</span>
<span class="cm-null"> </span>
<span class="cm-attribute">bar</span>
<span class="cm-null">=</span>
<span class="cm-string">"&quot;bar&quot;"</span>
<span class="cm-tag bracket">/></span>
<span class="cm-undefined">
</span>
<span class="cm-tag bracket"></</span>
<span class="cm-tag">foobar</span>
<span class="cm-tag bracket">></span>
<span class="cm-undefined">
</span>
</pre>

• Build artifacts shouldn't be checked into the repository—like lib/codemirror.js, they'll get created by the build process before publishing

Fixed by adding built runmode files to .gitignore.

• We support browsers that don't have Object.assign yet.

I initially tried to use the obj.copy in misc.js instead of Object.assign, but rollup disables the hasOwnProperty method on imported objects. Instead, I just iterate with a for loop.

• It would probably be easier to factor the runMode function into an ES6 file and import it from three different wrapper files to create the three versions of the addon

I initially tried this approach but rollup currently doesn't support custom UMD factory functions, that is there wasn't a clean non-hacky way of replacing the default generated header

(function (factory) {
  typeof define === 'function' && define.amd ? define(factory) :
  factory();
}((function () { 'use strict';

with the one used the current runmode.js,

(function(mod) {
  if (typeof exports == "object" && typeof module == "object") // CommonJS
    mod(require("../../lib/codemirror"));
  else if (typeof define == "function" && define.amd) // AMD
    define(["../../lib/codemirror"], mod);
  else // Plain browser env
    mod(CodeMirror);
})(function(CodeMirror) {

So I think appending addon/runmode/runmode.js to the end of the two autogenerated files, runmode-standalone and runmode.node is the best way to go.

• Either the liter needs to be configured to treat the content of addon/runmode/src as ES6, or those files could be moved to src/addon/runode

Moved to src/addon/runmode.

[marijnh]

rollup disables the hasOwnProperty method on imported objects

Object.prototype.hasOwnProperty.call works well in cases like this.

there wasn't a clean non-hacky way of replacing the default generated header

I'd be okay with just duplicating the environment detection in the runmode implementation, so that we don't have to mess with the UMD header. I'd much prefer a solution that uses rollup throughout to one that appends files.

[BrianHung]

Object.prototype.hasOwnProperty.call works well in cases like this.

That does work but would require either changing the copyObj function in misc.js or making a separate function.
https://github.com/codemirror/CodeMirror/blob/215c1dc644873190e3ad6aed77641017b10823e1/src/util/misc.js#L6-L12

I think iterating through the imported object and copying it to the CodeMirror or exports object achieves the same functionality.
https://github.com/codemirror/CodeMirror/blob/54f0f99c5d517ba4ced7126df86ed676b9aff8b3/src/addon/runmode/runmode-standalone.js#L10

I'd be okay with just duplicating the environment detection in the runmode implementation, so that we don't have to mess with the UMD header. I'd much prefer a solution that uses rollup throughout to one that appends files.

Sorry for the confusion here: we're still using rollup to autogenerate the entire file; it's because runmode.js doesn't have any dependencies to bundle up that the end effect is almost equal to appending it.

https://github.com/codemirror/CodeMirror/blob/54f0f99c5d517ba4ced7126df86ed676b9aff8b3/rollup.config.js#L24

[marijnh]

it's because runmode.js doesn't have any dependencies to bundle up that the end effect is almost equal to appending it.

Yes, maybe we're on the same page already, but what I meant was that I prefer for the file to be imported instead of appended.

[BrianHung]

Yes, maybe we're on the same page already, but what I meant was that I prefer for the file to be imported instead of appended.

I tried this approach initially to avoid multiple input files into rollup, but what I found was that regardless of where import "./addon/runmode/runmode.js" was placed, the immediately-invoked function within runmode would be bundled and ran before we defined the minimal CodeMirror.

So even if we had

import StringStream from "../../util/StringStream.js"
import * as modeMethods from "../../modes.js"

CodeMirror = {}
import "./addon/runmode/runmode.js"

rollup would produce

// imported StringStream and modeMethods
// immediately-invoked function requiring CodeMirror here
CodeMirror = {}

As a result, the function would error because CodeMirror hasn't been defined yet. The multi-entry plugin on the other-hand respects the order of input files, so it works.

The import way is still viable; however, this would require an additional file that acts as an import wrapper for each runmode-standalone and runmode.node.

// runmode-CodeMirror.js, creates the minimal CodeMirror needed for runmode
import StringStream from "../../util/StringStream.js"
import * as modeMethods from "../../modes.js"
CodeMirror = ...
export default CodeMirror

// single rollup entry point (wrapper)
import CodeMirror from "./runmode-CodeMirror.js"
import "./addon/runmode/runmode.js"

These two files produce the same bundled result that the multi-entry plugin creates. Which do you prefer?

Edit: The import wrapper way is included in the commit below.

[marijnh]

Merged, followed up with f87e118 and 6bb912b