Build improvements #2076

lbogdan · 2019-06-19T13:16:42Z

~~Description to come!~~

remove copy-webpack-plugin, replace it with

on dev, static serving, see

codesandbox-client/packages/app/scripts/start.js

Lines 192 to 196 in 0d5ef6b

    
           const rootPath = path.resolve(__dirname, '../../..'); 
        
           staticAssets.forEach(({ from, to }) => { 
        
             const fromPath = path.resolve(rootPath, from); 
        
             devServer.use(`/${to}`, express.static(fromPath)); 
        
           });

on prod, copying the static assets to the build folder in the newly added app/scripts/copy-assets.js; also move all gulp build logic to this script

remove happypack, as it's in maintenance mode, and use thread-loader instead
update webpack to latest 4.35.0
~~update webpack-dev-server to 3.2.1 (newer versions fail in a weird way, should maybe look more into that - reminder )~~
refactor code related to displaying warnings and errors in start.js and build.js, show elapsed (re)build times
disable HMR
small package.json scripts changes

L.E.:

(temporarily?) remove the auto-complete branch from CodeMirror, because it's using tern, which has acorn as a dependency, which fails to build, see https://drone.ops.csb.dev/codesandbox/codesandbox-client/281
we weren't loading the codemirror-tern chunk anyway, so I guess we were not using auto-complete? (although Ctrl-Space seems to work in the embeds?!)

Error in D:/work/webpack/codesandbox-client/node_modules/acorn/dist/acorn.es.js 3401:55
Module parse failed: Export 'parse_dammit' is not defined (3401:55)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
| }
|
> export { version, parse, parseExpressionAt, tokenizer, parse_dammit, LooseParser, pluginsLoose, addLooseExports, Parser, plugins, defaultOptions, Position, SourceLocation, getLineInfo, Node, TokenType, tt as tokTypes, keywordTypes, TokContext, types as tokContexts, isIdentifierChar, isIdentifierStart, Token, isNewLine, lineBreak, lineBreakG };
 @ D:/work/webpack/codesandbox-client/node_modules/tern/lib/tern.js 10:15-31
 @ ./src/app/components/CodeEditor/CodeMirror/index.js
[...]

L.L.E.:

remove unneeded gulp tasks

L.L.L.E.:

update webpack-dev-server to latest 3.7.2; starting with 3.3.0 (HMR plugin insertion for hot option and inline entry insertion in Server webpack/webpack-dev-server#1738) it auto-inserts the client into all entries based on the inline option, which by default is true, that's why we need

codesandbox-client/packages/app/scripts/start.js

Line 269 in 51ee807

inline: false,

because we selectively add it in

codesandbox-client/packages/app/config/webpack.dev.js

Lines 5 to 7 in bf0bd76

    
           const devEntries = [ 
        
             // 'react-hot-loader/patch', 
        
             'webpack-dev-server/client?/',

remove unused webpack-dev-middleware

Fixes #2069, #2070 .

CompuIves · 2019-06-19T15:44:14Z

packages/app/scripts/build.js

+        `Built ${stats.hasWarnings() ? 'with warnings ' : ''}in ${took /
+          1000}s.`
+      )
+    );
    console.log();

    // console.log('File sizes after gzip:');


We can start showing this again after we remove CopyWebpackPlugin, I commented this because it was also showing all files copied by the plugin (+50,000 files), which made the build actually slow because of slow stdout.

I think we can / should do something more involved as a separate step in the build process.

Hmm, like a check? I'd agree with that, if you can include it in this PR that would be nice, if not we could uncomment this for now in the meantime.

But how does that work, anyway, as in CI we don't have the previous build's assets?

We do! We cache them between builds in CircleCI.

… prod scripts.

CompuIves · 2019-06-20T08:51:21Z

packages/app/config/webpack.common.js

@@ -63,6 +50,12 @@ const ESLINT_PLUGIN_VUE_INDEX = `module.exports = {

 const sepRe = `\\${path.sep}`; // path separator regex

+const threadPoolConfig = {
+  workers: 2,


Did this make it faster than cpu().length - 1?

I didn't see any obvious improvements, even when using thread-loader vs. not using it, TBH. Can you maybe also test without it / with different numbers of workers and see what you get?

Didn't see improvements either, let's keep it at 2.

lbogdan · 2019-06-20T15:28:52Z

package.json

-      "**/react-codemirror2/**"
+      "**/react-codemirror2/**",
+      "app/webpack",
+      "app/acorn-dynamic-import"


(Hacky?) Fix for webpack/webpack#8656 until acornjs/acorn#834 gets merged.

…ev-middleware.

lbogdan · 2019-06-21T05:53:23Z

packages/app/scripts/start.js

@@ -51,7 +54,7 @@ function formatMessage(message) {
 function clearConsole() {
  // This seems to work best on Windows and other systems.
  // The intention is to clear the output so you can focus on most recent build.
-  process.stdout.write('\x1bc');
+  // process.stdout.write('\x1bc');


@CompuIves Should we completely remove clearConsole()? I kind of get its purpose, but on Windows all the previous output is completely lost, which is frustrating when you'd want to look at previous warnings / errors.

Hmm, I see. I liked the clearConsole but I think it makes sense to not do it in Windows. Maybe require('os').platform() === 'darwin' check?

Disabled by default on Windows and added CLEAR environment variable to override default behavior.

lbogdan · 2019-06-21T07:40:13Z

Whooo, everything's back to green! 😎

CompuIves · 2019-06-21T19:36:31Z

packages/app/src/app/components/CodeEditor/CodeMirror/index.js

+    //       cm.toggleComment({ lineComment: '//' });
+    //     });
+    //   },
+    //   'Cmd-P': () => {


Would it be possible to enable this one? That will allow the file fuzzy search

Re-added everything CodeMirror related back in.

- use CLEAR environment variable to override default behavior.

* [WIP] Build improvements. * Fix production build. * Extract static assets to config/build.js and use them in both dev and prod scripts. * Cleanup. * Update yarn.lock. * Update standalone-packages/vscode-textmate/package-lock.json. * Bump CircleCI cache keys version. * Revert "Update standalone-packages/vscode-textmate/package-lock.json." * Small improvements. * Update webpack to latest 4.35.0. * CodeMirror: remove auto-complete, so we're not bundling tern. * Remove unneeded gulp tasks. * Update webpack-dev-server to latest 3.7.2 and remove unused webpack-dev-middleware. * Show the updated module which triggers re-compiling. * Don't use thread-loader while linting. * Extract tern dep to codesandbox-deps and re-add CodeMirror bits. * Disable console clearing on Windows by default and - use CLEAR environment variable to override default behavior.

[WIP] Build improvements.

79916c2

lbogdan temporarily deployed to pr2076 June 19, 2019 13:24 Inactive

Fix production build.

057d47a

CompuIves reviewed Jun 19, 2019

View reviewed changes

lbogdan temporarily deployed to pr2076 June 19, 2019 15:48 Inactive

Extract static assets to config/build.js and use them in both dev and…

27145ec

… prod scripts.

lbogdan temporarily deployed to pr2076 June 19, 2019 17:16 Inactive

lbogdan added 5 commits June 19, 2019 20:19

Cleanup.

d0b7869

Update yarn.lock.

fd8d12e

Update standalone-packages/vscode-textmate/package-lock.json.

5ef8a8a

Bump CircleCI cache keys version.

b9de1f9

Revert "Update standalone-packages/vscode-textmate/package-lock.json."

f309784

lbogdan temporarily deployed to pr2076 June 19, 2019 17:52 Inactive

lbogdan temporarily deployed to pr2076 June 19, 2019 18:07 Inactive

Small improvements.

58e420f

lbogdan temporarily deployed to pr2076 June 20, 2019 07:52 Inactive

CompuIves reviewed Jun 20, 2019

View reviewed changes

lbogdan added 2 commits June 20, 2019 17:15

Update webpack to latest 4.35.0.

0d5ef6b

CodeMirror: remove auto-complete, so we're not bundling tern.

39fcae1

lbogdan force-pushed the experiment/build-improvements branch from f3681a2 to 39fcae1 Compare June 20, 2019 15:01

lbogdan temporarily deployed to pr2076 June 20, 2019 15:09 Inactive

lbogdan commented Jun 20, 2019

View reviewed changes

Remove unneeded gulp tasks.

31b3c20

lbogdan temporarily deployed to pr2076 June 20, 2019 16:32 Inactive

Update webpack-dev-server to latest 3.7.2 and remove unused webpack-d…

51ee807

…ev-middleware.

lbogdan temporarily deployed to pr2076 June 21, 2019 05:41 Inactive

lbogdan changed the title ~~[WIP] Build improvements~~ Build improvements Jun 21, 2019

lbogdan marked this pull request as ready for review June 21, 2019 05:46

lbogdan commented Jun 21, 2019

View reviewed changes

Show the updated module which triggers re-compiling.

a6877b2

lbogdan temporarily deployed to pr2076 June 21, 2019 06:16 Inactive

Don't use thread-loader while linting.

e7db9e6

lbogdan temporarily deployed to pr2076 June 21, 2019 07:11 Inactive

CompuIves reviewed Jun 21, 2019

View reviewed changes

Extract tern dep to codesandbox-deps and re-add CodeMirror bits.

25d4839

lbogdan temporarily deployed to pr2076 June 22, 2019 09:44 Inactive

Disable console clearing on Windows by default and

e142676

- use CLEAR environment variable to override default behavior.

lbogdan temporarily deployed to pr2076 June 22, 2019 10:20 Inactive

Merge branch 'master' into experiment/build-improvements

2ab9b69

lbogdan deployed to pr2076 June 22, 2019 11:19 View deployment

CompuIves merged commit 995a98a into codesandbox:master Jun 22, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build improvements #2076

Build improvements #2076

lbogdan commented Jun 19, 2019 •

edited

CompuIves Jun 19, 2019

lbogdan Jun 19, 2019

CompuIves Jun 19, 2019

lbogdan Jun 19, 2019 •

edited

CompuIves Jun 20, 2019

CompuIves Jun 20, 2019

lbogdan Jun 20, 2019

CompuIves Jun 21, 2019

lbogdan Jun 20, 2019 •

edited

lbogdan Jun 21, 2019

CompuIves Jun 21, 2019

lbogdan Jun 22, 2019 •

edited

lbogdan commented Jun 21, 2019

CompuIves Jun 21, 2019

lbogdan Jun 22, 2019

	const rootPath = path.resolve(__dirname, '../../..');
	staticAssets.forEach(({ from, to }) => {
	const fromPath = path.resolve(rootPath, from);
	devServer.use(`/${to}`, express.static(fromPath));
	});

	const devEntries = [
	// 'react-hot-loader/patch',
	'webpack-dev-server/client?/',

Build improvements #2076

Build improvements #2076

Conversation

lbogdan commented Jun 19, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lbogdan Jun 19, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lbogdan Jun 20, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lbogdan Jun 22, 2019 • edited

Choose a reason for hiding this comment

lbogdan commented Jun 21, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lbogdan commented Jun 19, 2019 •

edited

lbogdan Jun 19, 2019 •

edited

lbogdan Jun 20, 2019 •

edited

lbogdan Jun 22, 2019 •

edited