zlib: switch to lazy init for zlib streams #34048

puzpuzpuz · 2020-06-25T12:51:45Z

This PR is an experiment which hopefully mitigates #8871.

Moves zlib initialization to the point of execution on worker thread pool. This allows to postpone the corresponding memory allocation (the computation) to a later point in time which may be critical in situations when many zlib streams are created and operated simultaneously.

The following script leads to 2.7GB RSS on my machine with the latest master:

const zlib = require('zlib');

const message = {
  some: "data"
};
const payload = Buffer.from(JSON.stringify(message));

for (let i = 0; i < 30000; ++i) {
  zlib.deflate(payload, () => { });
}

setTimeout(() => { }, 2000000);

With the proposed changes, the RSS becomes around 200MB.

Benchmarks

https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/629/

18:55:08                                                                         confidence improvement accuracy (*)    (**)   (***)
18:55:08  zlib/createInflate.js n=100 chunkLen=1024 inputLen=16777216                     *     -5.26 %       ±5.13%  ±6.82%  ±8.88%
18:55:08  zlib/creation.js n=500000 options='false' type='BrotliCompress'                        2.22 %       ±5.93%  ±7.89% ±10.27%
18:55:08  zlib/creation.js n=500000 options='false' type='BrotliDecompress'                      3.36 %       ±5.58%  ±7.43%  ±9.67%
18:55:08  zlib/creation.js n=500000 options='false' type='Deflate'                      ***    322.44 %      ±10.82% ±14.57% ±19.31%
18:55:08  zlib/creation.js n=500000 options='false' type='DeflateRaw'                   ***    332.20 %      ±18.78% ±25.31% ±33.58%
18:55:08  zlib/creation.js n=500000 options='false' type='Gunzip'                       ***      9.90 %       ±4.47%  ±5.96%  ±7.77%
18:55:08  zlib/creation.js n=500000 options='false' type='Gzip'                         ***    326.31 %      ±17.46% ±23.51% ±31.19%
18:55:08  zlib/creation.js n=500000 options='false' type='Inflate'                      ***      9.83 %       ±5.18%  ±6.90%  ±8.99%
18:55:08  zlib/creation.js n=500000 options='false' type='InflateRaw'                     *      4.98 %       ±4.71%  ±6.27%  ±8.16%
18:55:08  zlib/creation.js n=500000 options='false' type='Unzip'                        ***      9.91 %       ±4.56%  ±6.07%  ±7.90%
18:55:08  zlib/creation.js n=500000 options='true' type='BrotliCompress'                         2.28 %       ±5.94%  ±7.91% ±10.31%
18:55:08  zlib/creation.js n=500000 options='true' type='BrotliDecompress'                       1.52 %       ±5.43%  ±7.23%  ±9.42%
18:55:08  zlib/creation.js n=500000 options='true' type='Deflate'                       ***    295.21 %      ±16.42% ±22.11% ±29.31%
18:55:08  zlib/creation.js n=500000 options='true' type='DeflateRaw'                    ***    298.23 %      ±14.62% ±19.69% ±26.12%
18:55:08  zlib/creation.js n=500000 options='true' type='Gunzip'                        ***     11.78 %       ±5.53%  ±7.37%  ±9.60%
18:55:08  zlib/creation.js n=500000 options='true' type='Gzip'                          ***    313.06 %      ±11.38% ±15.31% ±20.28%
18:55:08  zlib/creation.js n=500000 options='true' type='Inflate'                       ***      9.25 %       ±4.54%  ±6.05%  ±7.87%
18:55:08  zlib/creation.js n=500000 options='true' type='InflateRaw'                     **      6.93 %       ±5.20%  ±6.92%  ±9.01%
18:55:08  zlib/creation.js n=500000 options='true' type='Unzip'                         ***      8.22 %       ±4.49%  ±5.99%  ±7.81%
18:55:08  zlib/deflate.js n=400000 inputLen=1024 method='createDeflate'                         -4.22 %       ±9.00% ±11.98% ±15.59%
18:55:08  zlib/deflate.js n=400000 inputLen=1024 method='deflate'                         *      5.36 %       ±5.01%  ±6.70%  ±8.78%
18:55:08  zlib/deflate.js n=400000 inputLen=1024 method='deflateSync'                            1.33 %       ±7.70% ±10.25% ±13.34%
18:55:08  zlib/inflate.js n=400000 inputLen=1024 method='inflate'                               -1.71 %       ±3.17%  ±4.22%  ±5.52%
18:55:08  zlib/inflate.js n=400000 inputLen=1024 method='inflateSync'                           -5.23 %       ±5.60%  ±7.45%  ±9.70%
18:55:08  zlib/pipe.js algorithm='brotli' type='buffer' duration=5 inputLen=1024                -0.00 %       ±2.22%  ±2.95%  ±3.84%
18:55:08  zlib/pipe.js algorithm='brotli' type='string' duration=5 inputLen=1024                -0.34 %       ±1.43%  ±1.91%  ±2.49%
18:55:08  zlib/pipe.js algorithm='gzip' type='buffer' duration=5 inputLen=1024                   0.82 %       ±3.94%  ±5.24%  ±6.83%
18:55:08  zlib/pipe.js algorithm='gzip' type='string' duration=5 inputLen=1024                  -2.20 %       ±4.40%  ±5.86%  ±7.64%
18:55:09 
18:55:09 Be aware that when doing many comparisons the risk of a false-positive
18:55:09 result increases. In this case there are 28 comparisons, you can thus
18:55:09 expect the following amount of false-positive results:
18:55:09   1.40 false positives, when considering a   5% risk acceptance (*, **, ***),
18:55:09   0.28 false positives, when considering a   1% risk acceptance (**, ***),
18:55:09   0.03 false positives, when considering a 0.1% risk acceptance (***)

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

puzpuzpuz · 2020-06-25T18:48:26Z

@addaleax could you take a look at this one and tell me what you think?

addaleax

As noted in the issue, if this solves the problem, then great :) I don’t see anything speaking against going this route 👍

src/node_zlib.cc

addaleax

Thanks!

puzpuzpuz · 2020-06-26T13:03:50Z

@addaleax I've found that ResetStream() and SetParams() need to be called with properly initialized zlib. I have addressed that in 3e1c4833896d097fafeaf2668de7a054f9cb1162 and improved tests, so that they verify these changes. Please let me know if you see any issues in the update.

nodejs-github-bot · 2020-06-29T07:38:17Z

CI: https://ci.nodejs.org/job/node-test-pull-request/32140/

nodejs-github-bot · 2020-06-29T13:19:44Z

CI: https://ci.nodejs.org/job/node-test-pull-request/32146/

nodejs-github-bot · 2020-07-03T07:52:37Z

CI: https://ci.nodejs.org/job/node-test-pull-request/32195/

nodejs-github-bot · 2020-07-03T11:20:05Z

CI: https://ci.nodejs.org/job/node-test-pull-request/32196/

nodejs-github-bot · 2020-07-03T18:59:40Z

CI: https://ci.nodejs.org/job/node-test-pull-request/32206/

puzpuzpuz · 2020-07-04T06:33:36Z

I got a successful CI run, so formally this PR is mergeable. But as it's an experiment, I'd like to gather more opinions on this change.

cc @nodejs/zlib

yunyu · 2020-07-15T22:49:32Z

Hope you guys don't mind me asking, but is there a timeline for getting this merged? The zlib fragmentation issue is impacting our application ;(

PR-URL: #34499 Refs: #34048 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com>

Notable changes: dgram: * (SEMVER-MINOR) add IPv6 scope id suffix to received udp6 dgrams (Pekka Nikander) #14500 doc: * add AshCripps to collaborators (AshCripps) #34494 * add HarshithaKP to collaborators (Harshitha K P) #34417 * add rexagod to collaborators (Pranshu Srivastava) #34457 * add release key for Richard Lau (Richard Lau) #34397 events: * (SEMVER-MINOR) expand NodeEventTarget functionality (Anna Henningsen) #34057 src: * (SEMVER-MINOR) allow preventing SetPromiseRejectCallback (Shelley Vohr) #34387 * (SEMVER-MINOR) allow setting a dir for all diagnostic output (AshCripps) #33584 worker: * (SEMVER-MINOR) make MessagePort inherit from EventTarget (Anna Henningsen) #34057 zlib: * switch to lazy init for zlib streams (Andrey Pechkurov) #34048 PR-URL: #34542

Notable changes: deps: * upgrade npm to 6.14.7 (claudiahdz) #34468 dgram: * (SEMVER-MINOR) add IPv6 scope id suffix to received udp6 dgrams (Pekka Nikander) #14500 doc: * add AshCripps to collaborators (AshCripps) #34494 * add HarshithaKP to collaborators (Harshitha K P) #34417 * add rexagod to collaborators (Pranshu Srivastava) #34457 * add release key for Richard Lau (Richard Lau) #34397 events: * (SEMVER-MINOR) expand NodeEventTarget functionality (Anna Henningsen) #34057 src: * (SEMVER-MINOR) allow preventing SetPromiseRejectCallback (Shelley Vohr) #34387 * (SEMVER-MINOR) allow setting a dir for all diagnostic output (AshCripps) #33584 worker: * (SEMVER-MINOR) make MessagePort inherit from EventTarget (Anna Henningsen) #34057 zlib: * switch to lazy init for zlib streams (Andrey Pechkurov) #34048 PR-URL: #34542

RosenTomov · 2020-08-24T14:18:32Z

Hello, are there any plans to bring this fix over to the current LTS (Node 12)?

puzpuzpuz · 2020-08-24T14:25:56Z

@RosenTomov I wasn't planning to backport it, at least until I get some evidences that this patch mitigates the memory fragmentation issue for some users. Can you confirm that v14.7.0 mitigates the issue for you?

PR-URL: #34048 Reviewed-By: Anna Henningsen <anna@addaleax.net>

PR-URL: #34499 Refs: #34048 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com>

PR-URL: #34048 Reviewed-By: Anna Henningsen <anna@addaleax.net>

PR-URL: #34499 Refs: #34048 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com>

Notable changes: assert: * (SEMVER-MINOR) port common.mustCall() to assert (ConorDavenport) #31982 async_hooks: * (SEMVER-MINOR) add AsyncResource.bind utility (James M Snell) #34574 buffer: * (SEMVER-MINOR) also alias BigUInt methods (Anna Henningsen) #34960 * (SEMVER-MINOR) alias UInt ➡️ Uint in buffer methods (Anna Henningsen) #34729 build: * (SEMVER-MINOR) add build flag for OSS-Fuzz integration (davkor) #34761 cli: * (SEMVER-MINOR) add alias for report-directory to make it consistent (Ash Cripps) #33587 crypto: * (SEMVER-MINOR) allow KeyObjects in postMessage (Tobias Nießen) #33360 * (SEMVER-MINOR) add randomInt function (Oli Lalonde) #34600 deps: * upgrade to libuv 1.39.0 (Colin Ihrig) #34915 * upgrade npm to 6.14.7 (claudiahdz) #34468 * upgrade to libuv 1.38.1 (Colin Ihrig) #34187 dgram: * (SEMVER-MINOR) add IPv6 scope id suffix to received udp6 dgrams (Pekka Nikander) #14500 * (SEMVER-MINOR) allow typed arrays in .send() (Sarat Addepalli) #22413 doc: * (SEMVER-MINOR) Add maxTotalSockets option to agent constructor (rickyes) #33617 * (SEMVER-MINOR) add basic embedding example documentation (Anna Henningsen) #30467 * add Ricky Zhou to collaborators (rickyes) #34676 * add release key for Ruy Adorno (Ruy Adorno) #34628 * add DerekNonGeneric to collaborators (Derek Lewis) #34602 * add AshCripps to collaborators (Ash Cripps) #34494 * add HarshithaKP to collaborators (Harshitha K P) #34417 * add rexagod to collaborators (Pranshu Srivastava) #34457 * add release key for Richard Lau (Richard Lau) #34397 * add danielleadams to collaborators (Danielle Adams) #34360 * add sxa as collaborator (Stewart X Addison) #34338 * add ruyadorno to collaborators (Ruy Adorno) #34297 * (SEMVER-MAJOR) deprecate process.umask() with no arguments (Colin Ihrig) #32499 embedding: * (SEMVER-MINOR) make Stop() stop Workers (Anna Henningsen) #32531 * (SEMVER-MINOR) provide hook for custom process.exit() behaviour (Anna Henningsen) #32531 fs: * (SEMVER-MINOR) implement lutimes (Maël Nison) #33399 http: * (SEMVER-MINOR) add maxTotalSockets to agent class (rickyes) #33617 * (SEMVER-MINOR) return this from IncomingMessage#destroy() (Colin Ihrig) #32789 * (SEMVER-MINOR) expose host and protocol on ClientRequest (wenningplus) #33803 http2: * (SEMVER-MINOR) return this for Http2ServerRequest#setTimeout (Pranshu Srivastava) #33994 * (SEMVER-MINOR) do not modify explicity set date headers (Pranshu Srivastava) #33160 module: * (SEMVER-MINOR) named exports for CJS via static analysis (Guy Bedford) #35249 * (SEMVER-MINOR) exports pattern support (Guy Bedford) #34718 * (SEMVER-MINOR) package "imports" field (Guy Bedford) #34117 * (SEMVER-MINOR) deprecate module.parent (Antoine du HAMEL) #32217 n-api: * (SEMVER-MINOR) create N-API version 7 (Gabriel Schulhof) #35199 * (SEMVER-MINOR) support type-tagging objects (Gabriel Schulhof) #28237 n-api,src: * (SEMVER-MINOR) provide asynchronous cleanup hooks (Anna Henningsen) #34572 perf_hooks: * (SEMVER-MINOR) add idleTime and event loop util (Trevor Norris) #34938 timers: * (SEMVER-MINOR) allow timers to be used as primitives (Denys Otrishko) #34017 tls: * (SEMVER-MINOR) make 'createSecureContext' honor more options (Mateusz Krawczuk) #33974 worker: * (SEMVER-MINOR) add public method for marking objects as untransferable (Anna Henningsen) #33979 * (SEMVER-MINOR) emit `'messagerror'` events for failed deserialization (Anna Henningsen) #33772 * (SEMVER-MINOR) allow passing JS wrapper objects via postMessage (Anna Henningsen) #33772 * (SEMVER-MINOR) allow transferring/cloning generic BaseObjects (Anna Henningsen) #33772 * (SEMVER-MINOR) add option to track unmanaged file descriptors (Anna Henningsen) #34303 * (SEMVER-MINOR) add stack size resource limit option (Anna Henningsen) #33085 worker,fs: * (SEMVER-MINOR) make FileHandle transferable (Anna Henningsen) #33772 zlib: * (SEMVER-MINOR) add `maxOutputLength` option (unknown) #33516 * switch to lazy init for zlib streams (Andrey Pechkurov) #34048 PR-URL: TODO

Notable changes: assert: * (SEMVER-MINOR) port common.mustCall() to assert (ConorDavenport) #31982 async_hooks: * (SEMVER-MINOR) add AsyncResource.bind utility (James M Snell) #34574 buffer: * (SEMVER-MINOR) also alias BigUInt methods (Anna Henningsen) #34960 * (SEMVER-MINOR) alias UInt ➡️ Uint in buffer methods (Anna Henningsen) #34729 build: * (SEMVER-MINOR) add build flag for OSS-Fuzz integration (davkor) #34761 cli: * (SEMVER-MINOR) add alias for report-directory to make it consistent (Ash Cripps) #33587 crypto: * (SEMVER-MINOR) allow KeyObjects in postMessage (Tobias Nießen) #33360 * (SEMVER-MINOR) add randomInt function (Oli Lalonde) #34600 deps: * upgrade to libuv 1.39.0 (Colin Ihrig) #34915 * upgrade npm to 6.14.7 (claudiahdz) #34468 * upgrade to libuv 1.38.1 (Colin Ihrig) #34187 dgram: * (SEMVER-MINOR) add IPv6 scope id suffix to received udp6 dgrams (Pekka Nikander) #14500 * (SEMVER-MINOR) allow typed arrays in .send() (Sarat Addepalli) #22413 doc: * (SEMVER-MINOR) Add maxTotalSockets option to agent constructor (rickyes) #33617 * (SEMVER-MINOR) add basic embedding example documentation (Anna Henningsen) #30467 * add Ricky Zhou to collaborators (rickyes) #34676 * add release key for Ruy Adorno (Ruy Adorno) #34628 * add DerekNonGeneric to collaborators (Derek Lewis) #34602 * add AshCripps to collaborators (Ash Cripps) #34494 * add HarshithaKP to collaborators (Harshitha K P) #34417 * add rexagod to collaborators (Pranshu Srivastava) #34457 * add release key for Richard Lau (Richard Lau) #34397 * add danielleadams to collaborators (Danielle Adams) #34360 * add sxa as collaborator (Stewart X Addison) #34338 * add ruyadorno to collaborators (Ruy Adorno) #34297 * (SEMVER-MAJOR) deprecate process.umask() with no arguments (Colin Ihrig) #32499 embedding: * (SEMVER-MINOR) make Stop() stop Workers (Anna Henningsen) #32531 * (SEMVER-MINOR) provide hook for custom process.exit() behaviour (Anna Henningsen) #32531 fs: * (SEMVER-MINOR) implement lutimes (Maël Nison) #33399 http: * (SEMVER-MINOR) add maxTotalSockets to agent class (rickyes) #33617 * (SEMVER-MINOR) return this from IncomingMessage#destroy() (Colin Ihrig) #32789 * (SEMVER-MINOR) expose host and protocol on ClientRequest (wenningplus) #33803 http2: * (SEMVER-MINOR) return this for Http2ServerRequest#setTimeout (Pranshu Srivastava) #33994 * (SEMVER-MINOR) do not modify explicity set date headers (Pranshu Srivastava) #33160 module: * (SEMVER-MINOR) named exports for CJS via static analysis (Guy Bedford) #35249 * (SEMVER-MINOR) exports pattern support (Guy Bedford) #34718 * (SEMVER-MINOR) package "imports" field (Guy Bedford) #34117 * (SEMVER-MINOR) deprecate module.parent (Antoine du HAMEL) #32217 n-api: * (SEMVER-MINOR) create N-API version 7 (Gabriel Schulhof) #35199 * (SEMVER-MINOR) support type-tagging objects (Gabriel Schulhof) #28237 n-api,src: * (SEMVER-MINOR) provide asynchronous cleanup hooks (Anna Henningsen) #34572 perf_hooks: * (SEMVER-MINOR) add idleTime and event loop util (Trevor Norris) #34938 timers: * (SEMVER-MINOR) allow timers to be used as primitives (Denys Otrishko) #34017 tls: * (SEMVER-MINOR) make 'createSecureContext' honor more options (Mateusz Krawczuk) #33974 worker: * (SEMVER-MINOR) add public method for marking objects as untransferable (Anna Henningsen) #33979 * (SEMVER-MINOR) emit `'messagerror'` events for failed deserialization (Anna Henningsen) #33772 * (SEMVER-MINOR) allow passing JS wrapper objects via postMessage (Anna Henningsen) #33772 * (SEMVER-MINOR) allow transferring/cloning generic BaseObjects (Anna Henningsen) #33772 * (SEMVER-MINOR) add option to track unmanaged file descriptors (Anna Henningsen) #34303 * (SEMVER-MINOR) add stack size resource limit option (Anna Henningsen) #33085 worker,fs: * (SEMVER-MINOR) make FileHandle transferable (Anna Henningsen) #33772 zlib: * (SEMVER-MINOR) add `maxOutputLength` option (unknown) #33516 * switch to lazy init for zlib streams (Andrey Pechkurov) #34048 PR-URL: #35401

Notable changes: assert: * (SEMVER-MINOR) port common.mustCall() to assert (ConorDavenport) nodejs#31982 async_hooks: * (SEMVER-MINOR) add AsyncResource.bind utility (James M Snell) nodejs#34574 buffer: * (SEMVER-MINOR) also alias BigUInt methods (Anna Henningsen) nodejs#34960 * (SEMVER-MINOR) alias UInt ➡️ Uint in buffer methods (Anna Henningsen) nodejs#34729 build: * (SEMVER-MINOR) add build flag for OSS-Fuzz integration (davkor) nodejs#34761 cli: * (SEMVER-MINOR) add alias for report-directory to make it consistent (Ash Cripps) nodejs#33587 crypto: * (SEMVER-MINOR) allow KeyObjects in postMessage (Tobias Nießen) nodejs#33360 * (SEMVER-MINOR) add randomInt function (Oli Lalonde) nodejs#34600 deps: * upgrade to libuv 1.39.0 (Colin Ihrig) nodejs#34915 * upgrade npm to 6.14.7 (claudiahdz) nodejs#34468 * upgrade to libuv 1.38.1 (Colin Ihrig) nodejs#34187 dgram: * (SEMVER-MINOR) add IPv6 scope id suffix to received udp6 dgrams (Pekka Nikander) nodejs#14500 * (SEMVER-MINOR) allow typed arrays in .send() (Sarat Addepalli) nodejs#22413 doc: * (SEMVER-MINOR) Add maxTotalSockets option to agent constructor (rickyes) nodejs#33617 * (SEMVER-MINOR) add basic embedding example documentation (Anna Henningsen) nodejs#30467 * add Ricky Zhou to collaborators (rickyes) nodejs#34676 * add release key for Ruy Adorno (Ruy Adorno) nodejs#34628 * add DerekNonGeneric to collaborators (Derek Lewis) nodejs#34602 * add AshCripps to collaborators (Ash Cripps) nodejs#34494 * add HarshithaKP to collaborators (Harshitha K P) nodejs#34417 * add rexagod to collaborators (Pranshu Srivastava) nodejs#34457 * add release key for Richard Lau (Richard Lau) nodejs#34397 * add danielleadams to collaborators (Danielle Adams) nodejs#34360 * add sxa as collaborator (Stewart X Addison) nodejs#34338 * add ruyadorno to collaborators (Ruy Adorno) nodejs#34297 * (SEMVER-MAJOR) deprecate process.umask() with no arguments (Colin Ihrig) nodejs#32499 embedding: * (SEMVER-MINOR) make Stop() stop Workers (Anna Henningsen) nodejs#32531 * (SEMVER-MINOR) provide hook for custom process.exit() behaviour (Anna Henningsen) nodejs#32531 fs: * (SEMVER-MINOR) implement lutimes (Maël Nison) nodejs#33399 http: * (SEMVER-MINOR) add maxTotalSockets to agent class (rickyes) nodejs#33617 * (SEMVER-MINOR) return this from IncomingMessage#destroy() (Colin Ihrig) nodejs#32789 * (SEMVER-MINOR) expose host and protocol on ClientRequest (wenningplus) nodejs#33803 http2: * (SEMVER-MINOR) return this for Http2ServerRequest#setTimeout (Pranshu Srivastava) nodejs#33994 * (SEMVER-MINOR) do not modify explicity set date headers (Pranshu Srivastava) nodejs#33160 module: * (SEMVER-MINOR) named exports for CJS via static analysis (Guy Bedford) nodejs#35249 * (SEMVER-MINOR) exports pattern support (Guy Bedford) nodejs#34718 * (SEMVER-MINOR) package "imports" field (Guy Bedford) nodejs#34117 * (SEMVER-MINOR) deprecate module.parent (Antoine du HAMEL) nodejs#32217 n-api: * (SEMVER-MINOR) create N-API version 7 (Gabriel Schulhof) nodejs#35199 * (SEMVER-MINOR) support type-tagging objects (Gabriel Schulhof) nodejs#28237 n-api,src: * (SEMVER-MINOR) provide asynchronous cleanup hooks (Anna Henningsen) nodejs#34572 perf_hooks: * (SEMVER-MINOR) add idleTime and event loop util (Trevor Norris) nodejs#34938 timers: * (SEMVER-MINOR) allow timers to be used as primitives (Denys Otrishko) nodejs#34017 tls: * (SEMVER-MINOR) make 'createSecureContext' honor more options (Mateusz Krawczuk) nodejs#33974 worker: * (SEMVER-MINOR) add public method for marking objects as untransferable (Anna Henningsen) nodejs#33979 * (SEMVER-MINOR) emit `'messagerror'` events for failed deserialization (Anna Henningsen) nodejs#33772 * (SEMVER-MINOR) allow passing JS wrapper objects via postMessage (Anna Henningsen) nodejs#33772 * (SEMVER-MINOR) allow transferring/cloning generic BaseObjects (Anna Henningsen) nodejs#33772 * (SEMVER-MINOR) add option to track unmanaged file descriptors (Anna Henningsen) nodejs#34303 * (SEMVER-MINOR) add stack size resource limit option (Anna Henningsen) nodejs#33085 worker,fs: * (SEMVER-MINOR) make FileHandle transferable (Anna Henningsen) nodejs#33772 zlib: * (SEMVER-MINOR) add `maxOutputLength` option (unknown) nodejs#33516 * switch to lazy init for zlib streams (Andrey Pechkurov) nodejs#34048 PR-URL: nodejs#35401

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. zlib Issues and PRs related to the zlib subsystem. labels Jun 25, 2020

puzpuzpuz force-pushed the feature/zlib-lazy-init branch from 56eafab to 5dbd9f6 Compare June 25, 2020 18:28

puzpuzpuz marked this pull request as ready for review June 25, 2020 18:47

addaleax reviewed Jun 25, 2020

View reviewed changes

src/node_zlib.cc Outdated Show resolved Hide resolved

src/node_zlib.cc Show resolved Hide resolved

addaleax approved these changes Jun 26, 2020

View reviewed changes

puzpuzpuz mentioned this pull request Jun 26, 2020

zlib deflate results in a memory leak #8871

Open

This was referenced Jun 30, 2020

CI Relability 2020-06-30 JaneaSystems/nodejs-reliability#23

Open

CI Relability 2020-07-01 JaneaSystems/nodejs-reliability#24

Open

CI Relability 2020-07-02 JaneaSystems/nodejs-reliability#25

Open

CI Relability 2020-07-03 JaneaSystems/nodejs-reliability#26

Open

github-actions bot mentioned this pull request Jul 4, 2020

CI Relability 2020-07-04 JaneaSystems/nodejs-reliability#27

Open

32 tasks

github-actions bot mentioned this pull request Jul 12, 2020

CI Relability 2020-07-12 JaneaSystems/nodejs-reliability#35

Open

12 tasks

MylesBorins pushed a commit that referenced this pull request Jul 27, 2020

test: fix test-heapdump-zlib

3f11ba1

PR-URL: #34499 Refs: #34048 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com>

ruyadorno mentioned this pull request Jul 28, 2020

v14.7.0 proposal #34542

Merged

ruyadorno added the notable-change PRs with changes that should be highlighted in changelogs. label Jul 28, 2020

RosenTomov mentioned this pull request Aug 24, 2020

Memory leak socketio/socket.io#3477

Closed

addaleax pushed a commit that referenced this pull request Sep 22, 2020

zlib: switch to lazy init for zlib streams

720bda8

PR-URL: #34048 Reviewed-By: Anna Henningsen <anna@addaleax.net>

addaleax pushed a commit that referenced this pull request Sep 22, 2020

test: fix test-heapdump-zlib

aa576eb

PR-URL: #34499 Refs: #34048 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com>

addaleax pushed a commit that referenced this pull request Sep 22, 2020

zlib: switch to lazy init for zlib streams

d60b13f

PR-URL: #34048 Reviewed-By: Anna Henningsen <anna@addaleax.net>

addaleax pushed a commit that referenced this pull request Sep 22, 2020

test: fix test-heapdump-zlib

afd6e46

PR-URL: #34499 Refs: #34048 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com>

codebytere mentioned this pull request Sep 28, 2020

v12.19.0 proposal #35401

Merged

MylesBorins removed the baking-for-lts PRs that need to wait before landing in a LTS release. label Nov 24, 2020

tkurki mentioned this pull request Dec 1, 2021

wsCompression option unstable: memory issue SignalK/signalk-server#854

Open

mhdawson mentioned this pull request Jan 18, 2022

pummel/test-worker-take-heapsnapshot - crash in debug mode on arm #41204

Closed

kakaroto mentioned this pull request Feb 4, 2023

Compress world data to reduce loading time using socket compression options for large data transfer events. foundryvtt/foundryvtt#5942

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

zlib: switch to lazy init for zlib streams #34048

zlib: switch to lazy init for zlib streams #34048

puzpuzpuz commented Jun 25, 2020 •

edited

puzpuzpuz commented Jun 25, 2020

addaleax left a comment

addaleax left a comment

puzpuzpuz commented Jun 26, 2020

nodejs-github-bot commented Jun 29, 2020

nodejs-github-bot commented Jun 29, 2020

nodejs-github-bot commented Jul 3, 2020

nodejs-github-bot commented Jul 3, 2020

nodejs-github-bot commented Jul 3, 2020

puzpuzpuz commented Jul 4, 2020

yunyu commented Jul 15, 2020

RosenTomov commented Aug 24, 2020

puzpuzpuz commented Aug 24, 2020

zlib: switch to lazy init for zlib streams #34048

zlib: switch to lazy init for zlib streams #34048

Conversation

puzpuzpuz commented Jun 25, 2020 • edited

Benchmarks

Checklist

puzpuzpuz commented Jun 25, 2020

addaleax left a comment

Choose a reason for hiding this comment

addaleax left a comment

Choose a reason for hiding this comment

puzpuzpuz commented Jun 26, 2020

nodejs-github-bot commented Jun 29, 2020

nodejs-github-bot commented Jun 29, 2020

nodejs-github-bot commented Jul 3, 2020

nodejs-github-bot commented Jul 3, 2020

nodejs-github-bot commented Jul 3, 2020

puzpuzpuz commented Jul 4, 2020

yunyu commented Jul 15, 2020

RosenTomov commented Aug 24, 2020

puzpuzpuz commented Aug 24, 2020

puzpuzpuz commented Jun 25, 2020 •

edited