add wasm md4 implementation #14584

sokra · 2021-10-27T22:05:26Z

expose createHash is loader context

fixes #14532

What kind of change does this PR introduce?
bugfix

Did you add tests for your changes?
yes

Does this PR introduce a breaking change?
no

What needs to be documented once your changes are merged?
nothing

webpack-bot · 2021-10-27T22:05:30Z

For maintainers only:

This needs to be documented (issue in webpack/webpack.js.org will be filed when merged)
This needs to be backported to webpack 4 (issue will be created when merged)

expose createHash is loader context

webpack-bot · 2021-10-27T22:51:31Z

Thank you for your pull request! The most important CI builds succeeded, we’ll review the pull request soon.

thecrypticace · 2021-10-28T00:46:18Z

Passing note: I think this will need to be shared with / updated in loader-utils 2.x that used md4 in getHashDigest as well. I noticed that the Node 17 output.hashFunction fix doesn't work for when using css modules because of css-loader defaultGetLocalIdent -> loader-utils interpolateName -> loader-utils getHashDigest with md4.

vankop · 2021-10-28T07:19:50Z

assembly/hash/md4.asm.ts

+}
+
+function body(size: u32): void {
+	let _A = A;


just interesting what is a reason of this variables? (you can use A,B,C,D) same for a,b,c,d
using in current block reduce module size?

A is a global and _A a local var. I tried to avoid accessing globals too often

vankop · 2021-10-28T07:53:59Z

lib/util/hash/wasm-hash.js

+	digest(type) {
+		const { exports, buffered, mem, digestSize } = this;
+		exports.final(buffered);
+		this.instancesPool.push(this);


maybe instancesPool does not make much sense.. 64kb ram is a lot =)

If they are not pooled this would call new WebAssembly.Instance very often. I assumed that might be expensive, but I can try to benchmark if that's true.

alexander-akait · 2021-10-28T13:34:05Z

lib/NormalModule.js

@@ -539,7 +539,9 @@ class NormalModule extends Module {
 					? getContextifyInContext()(request)
 					: getContextify()(context, request);
 			},
-			createHash
+			createHash: type => {
+				return createHash(type || compilation.outputOptions.hashFunction);


When using latest node, webpack produces an error, Error: error:0308010C:digital envelope routines::unsupported webpack/webpack#14532 This is due to the webpack version being 5.53.0 when running npm install. The error is fixed in 5.61.0 by the following merged pull request, webpack/webpack#14584 https://github.com/webpack/webpack/releases/tag/v5.61.0 When I set 5.61.0 as the minimum I now get 5.72.0 version which is latest. Build completes without error.

webpack reports an error with OpenSSL 3 [1] as webpack uses md4, which is available only from the legacy provider. Newer webpack adds a wasm implementation of md4 [2], while buildbot uses very old webpack [3]. [1] webpack/webpack#14532 [2] webpack/webpack#14584 [3] https://github.com/buildbot/buildbot/blob/v3.6.1/www/build_common/package.json#L35 git-svn-id: file:///srv/repos/svn-community/svn@1342283 9fca08f4-af9d-4005-b8df-a31f2cc04f65

When using latest node, webpack produces an error, Error: error:0308010C:digital envelope routines::unsupported webpack/webpack#14532 This is due to the webpack version being 5.53.0 when running npm install. The error is fixed in 5.61.0 by the following merged pull request, webpack/webpack#14584 https://github.com/webpack/webpack/releases/tag/v5.61.0 When I set 5.61.0 as the minimum I now get 5.72.0 version which is latest. Build completes without error.

node 17 switched to openssl 3, which has some incompatibilities with older versions, see https://nodejs.org/en/blog/release/v17.0.0#openssl-3-0 webpack 4 relies on the older openssl version, only webpack 5 and newer uses openssl 3, see webpack/webpack#14532 (comment) webpack/webpack#14584 That's why we use the --openssl-legacy-provider node option when running webpack.

Node v17.0.0 changed to use OpenSSL 3.0 which no longer allows certain hashing algorithms by default. This breaks the version of webpack and, as a consequence, storybook we are currently using as it uses one of the now disallowed algorithms (MD4). This is fixed in later versions of webpack where support was added to generate MD4 hashs directly rather than via OpenSSL. To resolve this in webpack I have updated to the earliest version that contains their fix, v4.47.0. This version was chosen rather than a more recent version so we don't have to update all of our dependencies at this time. I have used a different approach for storybook as it would require considerable time and effort to update to a version that uses a fixed version of webpack. We just allow OpenSSL to use "legacy" algorithms using the NODE_OPTIONS=--openssl-legacy-provider environment variable. See for discussion: - https://stackoverflow.com/a/73027407 - webpack/webpack#14532 - https://nodejs.org/en/blog/release/v17.0.0#openssl-30 See for fix in webpack: - webpack/webpack#14584 - https://github.com/webpack/webpack/releases/tag/v4.47.0

* Resolve "error:0308010C:digital envelope routines::unsupported" Node v17.0.0 changed to use OpenSSL 3.0 which no longer allows certain hashing algorithms by default. This breaks the version of webpack and, as a consequence, storybook we are currently using as it uses one of the now-disallowed algorithms (MD4). This is fixed in later versions of webpack where support was added to generate MD4 hashs directly rather than via OpenSSL. To resolve this in webpack I have updated to the earliest version that contains their fix, v4.47.0. This version was chosen rather than a more recent version so we don't have to update all of our dependencies at this time. I have used a different approach for storybook as it would require considerable time and effort to update to a version that uses a fixed version of webpack. We just allow OpenSSL to use "legacy" algorithms using the NODE_OPTIONS=--openssl-legacy-provider environment variable. See for discussion: - https://stackoverflow.com/a/73027407 - webpack/webpack#14532 - https://nodejs.org/en/blog/release/v17.0.0#openssl-30 See for fix in webpack: - webpack/webpack#14584 - https://github.com/webpack/webpack/releases/tag/v4.47.0

When using latest node, webpack produces an error, Error: error:0308010C:digital envelope routines::unsupported webpack/webpack#14532 This is due to the webpack version being 5.53.0 when running npm install. The error is fixed in 5.61.0 by the following merged pull request, webpack/webpack#14584 https://github.com/webpack/webpack/releases/tag/v5.61.0 When I set 5.61.0 as the minimum I now get 5.72.0 version which is latest. Build completes without error.

webpack-bot added the PR: unreviewed label Oct 27, 2021

add wasm md4 implementation

dcf3b02

expose createHash is loader context

sokra force-pushed the hash/md4 branch from 308273c to dcf3b02 Compare October 27, 2021 22:07

massongit mentioned this pull request Oct 27, 2021

Bump node from 16.12.0-alpine3.12 to 17.0.1-alpine3.12 in /frontend dev-hato/hato-atama#508

Merged

webpack-bot added the PR: CI-ok label Oct 27, 2021

vankop reviewed Oct 28, 2021

View reviewed changes

improve hashing for longer strings, fix max short string length

ade40a4

alexander-akait mentioned this pull request Oct 28, 2021

nodejs 17: digital envelope routines::unsupported #14532

Closed

sokra added 4 commits October 28, 2021 12:08

improve micro benchmarks

7afc833

use correct batch size for BatchedHash

d806cf5

add support for additional digest types

205d3a0

default to the default hash function

a6bb3e5

alexander-akait reviewed Oct 28, 2021

View reviewed changes

vankop mentioned this pull request Oct 28, 2021

Dead store operation in xxhash64 code #14446

Closed

slorber mentioned this pull request Oct 28, 2021

feat: Support Node 17 facebook/docusaurus#5814

Merged

sokra merged commit 0f6c78c into main Oct 28, 2021

sokra deleted the hash/md4 branch October 28, 2021 20:58

gziolo mentioned this pull request Feb 11, 2022

Dependency Extraction Webpack Plugin fails on Node v17 WordPress/gutenberg#36471

Closed

nullterminated mentioned this pull request May 5, 2022

Update minimum version of webpack chibat/chrome-extension-typescript-starter#43

Merged

williamstein mentioned this pull request Aug 1, 2022

support building cocalc using nodejs version 18 sagemathinc/cocalc#6063

Closed

teppeis mentioned this pull request Aug 2, 2022

webpack throws an error with Node.js v18 cybozu/duck#740

Closed

pdehaan mentioned this pull request Mar 10, 2023

chore(deps): upgrade to node.js 18 mozilla/fxa#14985

Merged

astrojuanlu mentioned this pull request Mar 13, 2023

The react-scripts version locked for Kedro-Viz requires an insecure SSL setting on systems based on OpenSSL kedro-org/kedro-viz#1278

Closed

1 task

unflxw mentioned this pull request Jul 27, 2023

Error with recent Node versions (>=17) Migracode-Barcelona/react-hotel#1

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add wasm md4 implementation #14584

add wasm md4 implementation #14584

sokra commented Oct 27, 2021

webpack-bot commented Oct 27, 2021

webpack-bot commented Oct 27, 2021

thecrypticace commented Oct 28, 2021

vankop Oct 28, 2021 •

edited

sokra Oct 28, 2021

vankop Oct 28, 2021 •

edited

sokra Oct 28, 2021

alexander-akait Oct 28, 2021

add wasm md4 implementation #14584

add wasm md4 implementation #14584

Conversation

sokra commented Oct 27, 2021

webpack-bot commented Oct 27, 2021

webpack-bot commented Oct 27, 2021

thecrypticace commented Oct 28, 2021

vankop Oct 28, 2021 • edited

Choose a reason for hiding this comment

sokra Oct 28, 2021

Choose a reason for hiding this comment

vankop Oct 28, 2021 • edited

Choose a reason for hiding this comment

sokra Oct 28, 2021

Choose a reason for hiding this comment

alexander-akait Oct 28, 2021

Choose a reason for hiding this comment

vankop Oct 28, 2021 •

edited

vankop Oct 28, 2021 •

edited