Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib: add options to the heap snapshot APIs #44989

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
24 changes: 21 additions & 3 deletions doc/api/v8.md
Expand Up @@ -61,13 +61,23 @@ following properties:
}
```

## `v8.getHeapSnapshot()`
## `v8.getHeapSnapshot([options])`

<!-- YAML
added: v11.13.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/44989
description: Support options to configure the heap snapshot.
-->

* Returns: {stream.Readable} A Readable Stream containing the V8 heap snapshot
* `options` {Object}
* `exposeInternals` {boolean} If true, expose internals in the heap snapshot.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am actually not sure if we should use constants (e.g. strings) for these, suggestions are welcome. Also AFAIK exposeInternals doesn't currently change anything for us because we don't use any embedder tracing mechanism provided by V8 that could would make a difference here, but this might change in the future.

**Default:** `false`.
* `exposeNumericValues` {boolean} If true, expose numeric values in
artificial fields. **Default:** `false`.

* Returns: {stream.Readable} A Readable containing the V8 heap snapshot.

Generates a snapshot of the current V8 heap and returns a Readable
Stream that may be used to read the JSON serialized representation.
Expand Down Expand Up @@ -289,11 +299,14 @@ by [`NODE_V8_COVERAGE`][].
When the process is about to exit, one last coverage will still be written to
disk unless [`v8.stopCoverage()`][] is invoked before the process exits.

## `v8.writeHeapSnapshot([filename])`
## `v8.writeHeapSnapshot([filename[,options]])`

<!-- YAML
added: v11.13.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/44989
description: Support options to configure the heap snapshot.
- version: v18.0.0
pr-url: https://github.com/nodejs/node/pull/41373
description: An exception will now be thrown if the file could not be written.
Expand All @@ -308,6 +321,11 @@ changes:
generated, where `{pid}` will be the PID of the Node.js process,
`{thread_id}` will be `0` when `writeHeapSnapshot()` is called from
the main Node.js thread or the id of a worker thread.
* `options` {Object}
* `exposeInternals` {boolean} If true, expose internals in the heap snapshot.
**Default:** `false`.
* `exposeNumericValues` {boolean} If true, expose numeric values in
artificial fields. **Default:** `false`.
* Returns: {string} The filename where the snapshot was saved.

Generates a snapshot of the current V8 heap and writes it to a JSON
Expand Down
13 changes: 11 additions & 2 deletions doc/api/worker_threads.md
Expand Up @@ -1067,14 +1067,23 @@ added: v10.5.0
The `'online'` event is emitted when the worker thread has started executing
JavaScript code.

### `worker.getHeapSnapshot()`
### `worker.getHeapSnapshot([options])`

<!-- YAML
added:
- v13.9.0
- v12.17.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/44989
description: Support options to configure the heap snapshot.
-->

* `options` {Object}
* `exposeInternals` {boolean} If true, expose internals in the heap snapshot.
**Default:** `false`.
* `exposeNumericValues` {boolean} If true, expose numeric values in
artificial fields. **Default:** `false`.
* Returns: {Promise} A promise for a Readable Stream containing
a V8 heap snapshot

Expand Down Expand Up @@ -1379,7 +1388,7 @@ thread spawned will spawn another until the application crashes.
[`require('node:worker_threads').threadId`]: #workerthreadid
[`require('node:worker_threads').workerData`]: #workerworkerdata
[`trace_events`]: tracing.md
[`v8.getHeapSnapshot()`]: v8.md#v8getheapsnapshot
[`v8.getHeapSnapshot()`]: v8.md#v8getheapsnapshotoptions
[`vm`]: vm.md
[`worker.SHARE_ENV`]: #workershare_env
[`worker.on('message')`]: #event-message_1
Expand Down
19 changes: 17 additions & 2 deletions lib/internal/heap_utils.js
@@ -1,16 +1,30 @@
'use strict';
const {
Symbol
Symbol,
Uint8Array,
} = primordials;
const {
kUpdateTimer,
onStreamRead,
} = require('internal/stream_base_commons');
const { owner_symbol } = require('internal/async_hooks').symbols;
const { Readable } = require('stream');
const { validateObject, validateBoolean } = require('internal/validators');
const { kEmptyObject } = require('internal/util');

const kHandle = Symbol('kHandle');

function getHeapSnapshotOptions(options = kEmptyObject) {
validateObject(options, 'options');
const {
exposeInternals = false,
exposeNumericValues = false,
} = options;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to use destructuring + default values because assignment doesn't work with kEmtpyObject. @jasnell @aduh95

validateBoolean(exposeInternals, 'options.exposeInternals');
validateBoolean(exposeNumericValues, 'options.exposeNumericValues');
return new Uint8Array([+exposeInternals, +exposeNumericValues,]);
}

class HeapSnapshotStream extends Readable {
constructor(handle) {
super({ autoDestroy: true });
Expand All @@ -37,5 +51,6 @@ class HeapSnapshotStream extends Readable {
}

module.exports = {
HeapSnapshotStream
getHeapSnapshotOptions,
HeapSnapshotStream,
};
10 changes: 7 additions & 3 deletions lib/internal/worker.js
Expand Up @@ -416,12 +416,16 @@ class Worker extends EventEmitter {
return makeResourceLimits(this[kHandle].getResourceLimits());
}

getHeapSnapshot() {
const heapSnapshotTaker = this[kHandle] && this[kHandle].takeHeapSnapshot();
getHeapSnapshot(options) {
const {
HeapSnapshotStream,
getHeapSnapshotOptions
} = require('internal/heap_utils');
const optionsArray = getHeapSnapshotOptions(options);
const heapSnapshotTaker = this[kHandle]?.takeHeapSnapshot(optionsArray);
return new Promise((resolve, reject) => {
if (!heapSnapshotTaker) return reject(new ERR_WORKER_NOT_RUNNING());
heapSnapshotTaker.ondone = (handle) => {
const { HeapSnapshotStream } = require('internal/heap_utils');
resolve(new HeapSnapshotStream(handle));
};
});
Expand Down
23 changes: 18 additions & 5 deletions lib/v8.js
Expand Up @@ -57,31 +57,44 @@ const {
createHeapSnapshotStream,
triggerHeapSnapshot
} = internalBinding('heap_utils');
const { HeapSnapshotStream } = require('internal/heap_utils');
const {
HeapSnapshotStream,
getHeapSnapshotOptions
} = require('internal/heap_utils');
const promiseHooks = require('internal/promise_hooks');
const { getOptionValue } = require('internal/options');

/**
* Generates a snapshot of the current V8 heap
* and writes it to a JSON file.
* @param {string} [filename]
* @param {{
* exposeInternals?: boolean,
* exposeNumericValues?: boolean
* }} [options]
* @returns {string}
*/
function writeHeapSnapshot(filename) {
function writeHeapSnapshot(filename, options) {
if (filename !== undefined) {
filename = getValidatedPath(filename);
filename = toNamespacedPath(filename);
}
return triggerHeapSnapshot(filename);
const optionArray = getHeapSnapshotOptions(options);
return triggerHeapSnapshot(filename, optionArray);
}

/**
* Generates a snapshot of the current V8 heap
* and returns a Readable Stream.
* @param {{
* exposeInternals?: boolean,
* exposeNumericValues?: boolean
* }} [options]
* @returns {import('./stream.js').Readable}
*/
function getHeapSnapshot() {
const handle = createHeapSnapshotStream();
function getHeapSnapshot(options) {
const optionArray = getHeapSnapshotOptions(options);
const handle = createHeapSnapshotStream(optionArray);
assert(handle);
return new HeapSnapshotStream(handle);
}
Expand Down
6 changes: 5 additions & 1 deletion src/env.cc
Expand Up @@ -39,6 +39,7 @@ using v8::EscapableHandleScope;
using v8::Function;
using v8::FunctionTemplate;
using v8::HandleScope;
using v8::HeapProfiler;
using v8::HeapSpaceStatistics;
using v8::Integer;
using v8::Isolate;
Expand Down Expand Up @@ -1775,7 +1776,10 @@ size_t Environment::NearHeapLimitCallback(void* data,

Debug(env, DebugCategory::DIAGNOSTICS, "Start generating %s...\n", *name);

heap::WriteSnapshot(env, filename.c_str());
HeapProfiler::HeapSnapshotOptions options;
options.numerics_mode = HeapProfiler::NumericsMode::kExposeNumericValues;
options.snapshot_mode = HeapProfiler::HeapSnapshotMode::kExposeInternals;
heap::WriteSnapshot(env, filename.c_str(), options);
env->heap_limit_snapshot_taken_ += 1;

Debug(env,
Expand Down
47 changes: 35 additions & 12 deletions src/heap_utils.cc
Expand Up @@ -25,6 +25,7 @@ using v8::FunctionCallbackInfo;
using v8::FunctionTemplate;
using v8::Global;
using v8::HandleScope;
using v8::HeapProfiler;
using v8::HeapSnapshot;
using v8::Isolate;
using v8::JustVoid;
Expand All @@ -36,6 +37,7 @@ using v8::Number;
using v8::Object;
using v8::ObjectTemplate;
using v8::String;
using v8::Uint8Array;
using v8::Value;

namespace node {
Expand Down Expand Up @@ -340,15 +342,19 @@ class HeapSnapshotStream : public AsyncWrap,
HeapSnapshotPointer snapshot_;
};

inline void TakeSnapshot(Environment* env, v8::OutputStream* out) {
HeapSnapshotPointer snapshot {
env->isolate()->GetHeapProfiler()->TakeHeapSnapshot() };
inline void TakeSnapshot(Environment* env,
v8::OutputStream* out,
HeapProfiler::HeapSnapshotOptions options) {
HeapSnapshotPointer snapshot{
env->isolate()->GetHeapProfiler()->TakeHeapSnapshot(options)};
snapshot->Serialize(out, HeapSnapshot::kJSON);
}

} // namespace

Maybe<void> WriteSnapshot(Environment* env, const char* filename) {
Maybe<void> WriteSnapshot(Environment* env,
const char* filename,
HeapProfiler::HeapSnapshotOptions options) {
uv_fs_t req;
int err;

Expand All @@ -365,7 +371,7 @@ Maybe<void> WriteSnapshot(Environment* env, const char* filename) {
}

FileOutputStream stream(fd, &req);
TakeSnapshot(env, &stream);
TakeSnapshot(env, &stream, options);
if ((err = stream.status()) < 0) {
env->ThrowUVException(err, "write", nullptr, filename);
return Nothing<void>();
Expand Down Expand Up @@ -410,10 +416,28 @@ BaseObjectPtr<AsyncWrap> CreateHeapSnapshotStream(
return MakeBaseObject<HeapSnapshotStream>(env, std::move(snapshot), obj);
}

HeapProfiler::HeapSnapshotOptions GetHeapSnapshotOptions(
Local<Value> options_value) {
CHECK(options_value->IsUint8Array());
Local<Uint8Array> arr = options_value.As<Uint8Array>();
uint8_t* options =
static_cast<uint8_t*>(arr->Buffer()->Data()) + arr->ByteOffset();
HeapProfiler::HeapSnapshotOptions result;
result.snapshot_mode = options[0]
? HeapProfiler::HeapSnapshotMode::kExposeInternals
: HeapProfiler::HeapSnapshotMode::kRegular;
result.numerics_mode = options[1]
? HeapProfiler::NumericsMode::kExposeNumericValues
: HeapProfiler::NumericsMode::kHideNumericValues;
return result;
}

void CreateHeapSnapshotStream(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
HeapSnapshotPointer snapshot {
env->isolate()->GetHeapProfiler()->TakeHeapSnapshot() };
CHECK_EQ(args.Length(), 1);
auto options = GetHeapSnapshotOptions(args[0]);
HeapSnapshotPointer snapshot{
env->isolate()->GetHeapProfiler()->TakeHeapSnapshot(options)};
CHECK(snapshot);
BaseObjectPtr<AsyncWrap> stream =
CreateHeapSnapshotStream(env, std::move(snapshot));
Expand All @@ -424,13 +448,13 @@ void CreateHeapSnapshotStream(const FunctionCallbackInfo<Value>& args) {
void TriggerHeapSnapshot(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
Isolate* isolate = args.GetIsolate();

CHECK_EQ(args.Length(), 2);
Local<Value> filename_v = args[0];
auto options = GetHeapSnapshotOptions(args[1]);

if (filename_v->IsUndefined()) {
DiagnosticFilename name(env, "Heap", "heapsnapshot");
if (WriteSnapshot(env, *name).IsNothing())
return;
if (WriteSnapshot(env, *name, options).IsNothing()) return;
if (String::NewFromUtf8(isolate, *name).ToLocal(&filename_v)) {
args.GetReturnValue().Set(filename_v);
}
Expand All @@ -439,8 +463,7 @@ void TriggerHeapSnapshot(const FunctionCallbackInfo<Value>& args) {

BufferValue path(isolate, filename_v);
CHECK_NOT_NULL(*path);
if (WriteSnapshot(env, *path).IsNothing())
return;
if (WriteSnapshot(env, *path, options).IsNothing()) return;
return args.GetReturnValue().Set(filename_v);
}

Expand Down
10 changes: 9 additions & 1 deletion src/node_internals.h
Expand Up @@ -380,7 +380,9 @@ class DiagnosticFilename {
};

namespace heap {
v8::Maybe<void> WriteSnapshot(Environment* env, const char* filename);
v8::Maybe<void> WriteSnapshot(Environment* env,
const char* filename,
v8::HeapProfiler::HeapSnapshotOptions options);
}

namespace heap {
Expand Down Expand Up @@ -421,6 +423,12 @@ std::ostream& operator<<(std::ostream& output,
}

bool linux_at_secure();

namespace heap {
v8::HeapProfiler::HeapSnapshotOptions GetHeapSnapshotOptions(
v8::Local<v8::Value> options);
} // namespace heap

} // namespace node

#endif // defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS
Expand Down
8 changes: 5 additions & 3 deletions src/node_worker.cc
Expand Up @@ -778,6 +778,8 @@ class WorkerHeapSnapshotTaker : public AsyncWrap {
void Worker::TakeHeapSnapshot(const FunctionCallbackInfo<Value>& args) {
Worker* w;
ASSIGN_OR_RETURN_UNWRAP(&w, args.This());
CHECK_EQ(args.Length(), 1);
auto options = heap::GetHeapSnapshotOptions(args[0]);

Debug(w, "Worker %llu taking heap snapshot", w->thread_id_.id);

Expand All @@ -797,10 +799,10 @@ void Worker::TakeHeapSnapshot(const FunctionCallbackInfo<Value>& args) {

// Interrupt the worker thread and take a snapshot, then schedule a call
// on the parent thread that turns that snapshot into a readable stream.
bool scheduled = w->RequestInterrupt([taker = std::move(taker),
env](Environment* worker_env) mutable {
bool scheduled = w->RequestInterrupt([taker = std::move(taker), env, options](
Environment* worker_env) mutable {
heap::HeapSnapshotPointer snapshot{
worker_env->isolate()->GetHeapProfiler()->TakeHeapSnapshot()};
worker_env->isolate()->GetHeapProfiler()->TakeHeapSnapshot(options)};
CHECK(snapshot);

// Here, the worker thread temporarily owns the WorkerHeapSnapshotTaker
Expand Down