Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

diagnostics_channel: fix diagnostics channel memory leak #45633

Merged

Conversation

theanarkh
Copy link
Contributor

@theanarkh theanarkh commented Nov 26, 2022

fix diagnostics channel memory leak.
cc @Qard

const { subscribe, unsubscribe } = require('diagnostics_channel');

function noop() {}

console.log(process.memoryUsage().heapUsed);

for (let i = 0; i < 1000000; i++) {
    subscribe(String(i), noop);
    unsubscribe(String(i), noop);
}

gc();

console.log(process.memoryUsage().heapUsed);
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Nov 26, 2022
@theanarkh theanarkh force-pushed the fix_diagnostics_channel_memory_leak branch from ae3c2f9 to 2026e9c Compare November 26, 2022 19:21
@Qard
Copy link
Member

Qard commented Nov 26, 2022

LGTM. Not really how users are intended to use diagnostics_channel, but how users are meant to use things and how they actually use them doesn't always match, so seems like a reasonable fix to me. 😅

Copy link
Member

@anonrig anonrig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@theanarkh theanarkh added the request-ci Add this label to start a Jenkins CI on a PR. label Nov 27, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Nov 27, 2022
@nodejs-github-bot
Copy link
Collaborator

@RaisinTen
Copy link
Contributor

Could you add a test for this? Maybe by comparing the values of process.memoryUsage().heapUsed before and after the loop and gc call?

@theanarkh
Copy link
Contributor Author

Could you add a test for this? Maybe by comparing the values of process.memoryUsage().heapUsed before and after the loop and gc call?

I think it is difficult to do that, do you have any idea? Thanks!

@RaisinTen
Copy link
Contributor

I think it is difficult to do that, do you have any idea? Thanks!

I meant using the code you shared in the PR description with a slight modification - asserting that the value of process.memoryUsage().heapUsed after the loop+gc is less than or equal to the value before. Like this:

// Flags: --expose-gc
'use strict';

// This test ensures that diagnostic channel references aren't leaked.

require('../common');
const { ok } = require('assert');

const { subscribe, unsubscribe } = require('diagnostics_channel');

function noop() {}

const heapUsedBefore = process.memoryUsage().heapUsed;

for (let i = 0; i < 1000000; i++) {
    subscribe(String(i), noop);
    unsubscribe(String(i), noop);
}

gc();

const heapUsedAfter = process.memoryUsage().heapUsed;

ok(heapUsedBefore >= heapUsedAfter);

Is it difficult because it's too flaky?

@theanarkh theanarkh force-pushed the fix_diagnostics_channel_memory_leak branch from 2026e9c to cf83822 Compare November 28, 2022 14:37
@theanarkh
Copy link
Contributor Author

theanarkh commented Nov 28, 2022

Thanks for the suggestion. When i add the test(i = 1000000), i find the output is as follows.

before gc 5350312
after  gc 15322320

But when i change the code channels[name] = null; to delete channels[name];, the output is as follows.

before gc 5353152
after  gc 4639440

Then i take the snapshot before and after the loop. I find It takes up ten more megabytes of memory in channels[name] = null;(But when i is 100, the output of two cases is the same) 😄.
image

@theanarkh theanarkh force-pushed the fix_diagnostics_channel_memory_leak branch from cf83822 to 4fb75ce Compare November 28, 2022 15:02
Copy link
Contributor

@RaisinTen RaisinTen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@theanarkh theanarkh added the request-ci Add this label to start a Jenkins CI on a PR. label Nov 29, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Nov 29, 2022
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@theanarkh theanarkh added the commit-queue Add this label to land a pull request using GitHub Actions. label Nov 29, 2022
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Nov 29, 2022
@nodejs-github-bot nodejs-github-bot merged commit d579bc1 into nodejs:main Nov 29, 2022
@nodejs-github-bot
Copy link
Collaborator

Landed in d579bc1

ErickWendel pushed a commit to ErickWendel/node that referenced this pull request Nov 30, 2022
PR-URL: nodejs#45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
targos pushed a commit that referenced this pull request Dec 12, 2022
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Dec 30, 2022
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Dec 30, 2022
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Jan 3, 2023
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Jan 4, 2023
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Jan 5, 2023
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants