module: add support for abi stable module API #11975

jasongin · 2017-03-21T19:25:21Z

Add support for abi stable module API (N-API) as "Experimental feature".
The goal of this API is to provide a stable Node API for native
module developers. N-API aims to provide ABI compatibility guarantees
across different Node versions and also across different
Node VMs - allowing N-API enabled native modules to just work
across different versions and flavors of Node.js without recompilation.

A more detailed introduction is provided in:
https://github.com/nodejs/node-eps/blob/master/005-ABI-Stable-Module-API.md
and https://github.com/nodejs/abi-stable-node/blob/doc/VM%20Summit.pdf.

The feature, during its experimental state, will be guarded by a runtime
flag "--napi-modules". Only when this flag is added to the command line
will N-API modules along with regular non N-API modules be supported.

The API is defined by the methods in src/node_api.h,
src/node_api_types.h and "src/node_api_async.h". This is the best
starting point to review the API surface. More documentation will follow.

In addition to the implementation of the API using V8, which is included
in this PR, the API has also been validated against chakracore and that
port is available in
https://github.com/nodejs/abi-stable-node/tree/api-prototype-chakracore-8.x.

The current plan is to provide N-API support in versions 8.X and 6.X
directly. For older versions, such as 4.X or pre N-API versions of 6.X,
we plan to create an external npm module to provide a migration path
that will allow modules targeting older Node.js versions to use the API,
albeit without getting the advantage of not having to recompile.

In addition, the external npm package will include C++ sugar to
simplify the use of the API. The sugar will be in-line only and will
only use the exported N-API methods but is not part of the N-API
itself. The current version is in:
https://github.com/nodejs/node-api.

This PR is a result of work in the abi-stable-node repo:
https://github.com/nodejs/abi-stable-node/tree/doc,
with this PR being the cumulative work on the api-prototype-8.x
branch with the following contributors in alphabetical order:

Arunesh Chandra (@aruneshchandra)
Gabriel Schulhof (@gabrielschulhof)
Hitesh Kanwathirtha (@digitalinfinity)
Ian Halliday (@ianwjhalliday)
Jason Ginchereau (@jasongin)
Michael Dawson (@mhdawson)
Sampson Gao (@sampsongao)
Taylor Woll (@boingoing)

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added - IN PROGRESS
commit message follows commit guidelines

Affected core subsystem(s)

module

@aruneshchandra

Add support for abi stable module API (N-API) as "Experimental feature". The goal of this API is to provide a stable Node API for native module developers. N-API aims to provide ABI compatibility guarantees across different Node versions and also across different Node VMs - allowing N-API enabled native modules to just work across different versions and flavors of Node.js without recompilation. A more detailed introduction is provided in: https://github.com/nodejs/node-eps/blob/master/005-ABI-Stable-Module-API.md and https://github.com/nodejs/abi-stable-node/blob/doc/VM%20Summit.pdf. The feature, during its experimental state, will be guarded by a runtime flag "--napi-modules". Only when this flag is added to the command line will N-API modules along with regular non N-API modules be supported. The API is defined by the methods in "src/node_api.h", "src/node_api_types.h" and "src/node_api_async.h". This is the best starting point to review the API surface. More documentation will follow. In addition to the implementation of the API using V8, which is included in this PR, the API has also been validated against chakracore and that port is available in https://github.com/nodejs/abi-stable-node/tree/api-prototype-chakracore-8.x. The current plan is to provide N-API support in versions 8.X and 6.X directly. For older versions, such as 4.X or pre N-API versions of 6.X, we plan to create an external npm module to provide a migration path that will allow modules targeting older Node.js versions to use the API, albeit without getting the advantage of not having to recompile. In addition, we also plan an external npm package with C++ sugar to simplify the use of the API. The sugar will be in-line only and will only use the exported N-API methods but is not part of the N-API itself. The current version is in: https://github.com/nodejs/node-api. This PR is a result of work in the abi-stable-node repo: https://github.com/nodejs/abi-stable-node/tree/doc, with this PR being the cumulative work on the api-prototype-8.x branch with the following contributors in alphabetical order: Arunesh Chandra (@aruneshchandra) Gabriel Schulhof (@gabrielschulhof) Hitesh Kanwathirtha (@digitalinfinity) Ian Halliday (@ianwjhalliday) Jason Ginchereau (@jasongin) Michael Dawson (@mhdawson) Sampson Gao (@sampsongao) Taylor Woll (@boingoing)

richardlau · 2017-03-21T19:49:09Z

src/node.cc

@@ -3535,6 +3547,7 @@ static void PrintHelp() {
         "  --trace-deprecation        show stack traces on deprecations\n"
         "  --throw-deprecation        throw an exception on deprecations\n"
         "  --no-warnings              silence all process warnings\n"
+         "  --napi-modules[=yes|no]    load N-API modules (default no)\n"


As the comment at the beginning of this function says:

// XXX: If you add an option here, please also add it to doc/node.1 and // doc/api/cli.md

I think this would be our only option that has a [=yes|no] format (apart from configure flags, I guess). Is there a reason for that?

If somebody really wants to avoid loading N-API modules even after we've eventually set the default to "yes", thus rendering the use of the command line option unnecessary, they will be unable to do so unless we give them the option to say "no".

Of course, if the consensus is that we do not need to provide users with a way to explicitly disable the support for N-API modules, then the option can be reducded to --napi-modules.

If somebody really wants to avoid loading N-API modules even after we've eventually set the default to "yes"

I'm not sure that's something that we'll need to support at that point. If N-API is successful it should become the normal way of writing native modules, so it wouldn't really make sense to disable it.

I agree. Once non-experimental we would would most likely just remove the flag. I think at this point we could just likely make it --napi-modules with no options.

I personally would prefer --napi-modules and then once it goes live we could add an optional =false or something to force-disable?

@Fishrock123 can you explain why you (or someone) might want to disable the feature after it is out of experimental status?

No, actually. I really do not know, but I would prefer to not have =yes.

Resolved by 87c42e7

trevnorris

Quick review done. Great work on the tests. Will give more thorough review later, but only have a few small comments for now.

trevnorris · 2017-03-21T20:46:43Z

src/node_api.h

+NAPI_EXTERN napi_status napi_create_string_utf16(napi_env env,
+                                                 const char16_t* s,
+                                                 int length,
+                                                 napi_value* result);


why no napi_create_string_latin1? That's a fairly important type for string conversion.

Good feedback, we should consider adding it. I opened nodejs/abi-stable-node#179 to track this.

Note missing APIs should probably not block this PR unless it is something critical. The goal for this PR is to establish an initial set of APIs that provide enough functionality for native module developers to begin experimenting and give us feedback. We have plans to expand on these APIs in the coming months, before the feature graduates out of experimental status.

As @jasongin mentioned, we don't believe missing APIs should be a blocker. There are a few others we have in our plans to add which we will prioritized based on feedback from people using the API.

trevnorris · 2017-03-21T20:51:56Z

src/node_api.cc

+}
+
+napi_status napi_escape_handle(napi_env e,
+


Why not use v8::Object::GetPrototype() here?

Did you mean to put this comment somewhere else?

I'm guessing you might be referring to the access of the prototype property in napi_instanceof() here. In that case, it is really a function's prototype property that is needed, that is different from the __proto__ property that is returned by GetPrototype(). The "instanceof" logic had to be implemented in this way rather than using v8::FunctionTemplate::HasInstance() because N-API does not expose the V8-specific concept of function or object templates, nor does it track them internally, so the FunctionTemplate that was used to create the function passed to napi_instanceof() is not available at that point.

(There is also a napi_get_prototype() API that simply calls v8::Object::GetPrototype().)

ah yup. see what you're saying.

trevnorris · 2017-03-21T21:11:34Z

src/node_api.cc

+  RETURN_STATUS_IF_FALSE(v8impl::TryCatch::lastException().IsEmpty(), \
+                         napi_pending_exception);                     \
+  napi_clear_last_error();                                            \
+  v8impl::TryCatch tryCatch(v8impl::V8IsolateFromJsEnv((e)))


Looks like NAP_PREAMBLE is used in many functions that don't execute javascript. what's the rational for using TryCatch when no JS is called and the v8 Maybe APIs are used?

We have made an effort to avoid using NAPI_PREAMBLE for functions which don't execute JavaScript (see napi_is_error for example) but we haven't done that exhaustively. I think we took the more conservative approach of including TryCatch when we weren't sure if it was strictly required or not because the guidance seems to be that using TryCatch was cheap enough that this would be fine. If we're sure no JavaScript is getting executed, removing the NAPI_PREAMBLE seems like the right thing to do.

Also we know we need to remove the NAPI_PREAMBLE from at least a few of the functions, for https://github.com/nodejs/abi-stable-node/issues/122

Thanks for the explanation. To clarify, I don't see this as a blocker.

trevnorris · 2017-03-21T21:20:46Z

src/node_api.cc

+  } else if (finalize_cb != nullptr) {
+    // Create a self-deleting reference just for the finalize callback.
+    new v8impl::Reference(
+        isolate, obj, 0, true, finalize_cb, nativeObj, finalize_hint);


Not sure I can appreciate this auto-magical deletion of the the pointer. One reason is because there's no way to know whether the pointer was made via new or malloc. Even in node_base_object.h there's an explicit call to make the object weak.

Edit: missed the "self-deleting" part. Nothing to see here.

trevnorris · 2017-03-21T21:22:45Z

src/node_api.cc

+    }
+
+    if (deleteSelf) {
+      delete reference;


It feels counter intuitive to go to such lengths to make the public API be C, but then use delete on a user-supplied pointer.

The delete here is deleting our Reference instance - not the user-supplied pointer. We aren't (hopefully) doing anything which would preclude a module written in C.

There is another delete reference in napi_delete_reference which operates on a user-supplied pointer (the napi_ref argument). We are expecting the user to hand us back a Reference* which we allocated (via new) in napi_create_reference. This Reference class acts as a container for us to do ref-counting.

If the user code calls napi_delete_reference with any random pointer, I guess behavior would be undefined. We would try to use ~Reference which might crash. But this is true of all the API which take an opaque pointer and cast it to an engine-specific type. We have a similar pattern for napi_close_handle_scope and napi_close_escapable_handle_scope.

Yeah, the code looks correct. I’d say for these cases it would be nice to turn the constructor private and re-expose it as a static method (New() or something), so that it’s more obvious that the instance was allocated using new. (Plus, imho it’s good practice to try and keep corresponding new and delete calls reasonably close to each other)

@trevnorris it sounds like we should have a test where an addon is implemented in C, just to keep us honest.

fb8ced4 adds New() and Delete() methods on the Reference class for clarity.

my bad. misunderstood what was going on here.

The snprintf API is not available with older versions of MSVC. Found by a CI run.

addaleax · 2017-03-22T03:10:10Z

Btw, in ba4847e we listed the authors as Author: name <email> as PR metadata, it looks like doing the same thing might make sense here, too?

addaleax · 2017-03-22T03:14:37Z

src/node.cc

+      snprintf(errmsg,
+               sizeof(errmsg),
+               "The module '%s'"
+               "\nwas compiled against the Node.js API. This feature is "


This error message could be a bit more specific than the Node.js API, I’d say. Also, maybe display the flag name here directly?

Resolved by b129624

addaleax · 2017-03-22T03:15:43Z

src/node.cc

@@ -3535,6 +3547,7 @@ static void PrintHelp() {
         "  --trace-deprecation        show stack traces on deprecations\n"
         "  --throw-deprecation        throw an exception on deprecations\n"
         "  --no-warnings              silence all process warnings\n"
+         "  --napi-modules[=yes|no]    load N-API modules (default no)\n"


I think this would be our only option that has a [=yes|no] format (apart from configure flags, I guess). Is there a reason for that?

addaleax · 2017-03-22T03:17:10Z

src/node.cc

+    ProcessEmitWarning(&env, "N-API is an experimental feature "
+        "and could change at any time.");
+  }
+


I’m not a fan of printing warnings like these… if the user opted into the feature, then they opted in because they know what they were asking for.

CTC has specifically requested command-line enablement along with a warning message for features with experimental status. Tracing does the same thing.

Is there a link somewhere to that decision?

I don't quite understand the justification for a printed warning for a feature that can't even be used without a flag. If you're using the flag, you wanted the feature and are quite aware that it is experimental. Do any V8 features do this?

VM Summit 3 meeting notes has this
For 8.0, include an opt-in runtime flag, show experimental warning when a NAPI module is loaded.

If consensus is that documentation of experimental status is sufficient and this warning message is not necessary, we can take it out. I don't recall anyone feeling very strongly about this at the VM summit.

@jasongin I’d say take it out, and we can re-visit that in the unlikely case anybody complains.

I think the main motivation was to ensure that anybody turning it on is fully aware of the status and to be consistent with what we have done for other experimental features.

addaleax · 2017-03-22T03:20:51Z

src/node_api.cc

+    v8::Local<v8::Value> l;
+    U(v8::Local<v8::Value> _l) : l(_l) {}
+  } u(local);
+  assert(sizeof(u.v) == sizeof(u.l));


static_assert for all of these

Resolved by 116bd19

addaleax · 2017-03-22T03:27:38Z

src/node_api.cc

+  } u(local);
+  assert(sizeof(u.v) == sizeof(u.l));
+  return u.v;
+}


Yeah, awkward is a good description, I’m not even sure these would work if we didn’t compile with -fno-strict-aliasing. Is there anything speaking against a simple return reinterpret_cast<napi_value>(*local), at least for the V8-to-napi_value conversion?

return reinterpret_cast<napi_value>(*local)

Yes, I agree that's correct and more straightforward. I'll update it.

(This is some of the oldest code in the N-API project; I'm guessing whoever wrote it was initially trying to avoid using reinterpret_cast<>, though the union accomplishes the same thing in a roundabout way.)

Resolved by 116bd19

addaleax · 2017-03-22T04:19:14Z

src/node_api.cc

+// Gets the number of CHARACTERS in the string.
+napi_status napi_get_value_string_length(napi_env e,
+                                         napi_value value,
+                                         int* result) {


It seems like int is a V8 implementation detail here too, size_t is probably better for string lengths (also here and below)?

Yes, good point.

Resolved by 9e3ab83

addaleax · 2017-03-22T04:20:01Z

src/node_api.cc

+}
+
+napi_status napi_escape_handle(napi_env e,
+


It seems to me like int is a V8 implementation detail, size_t might better for argument counts (here and elsewhere)?

Resolved by 9e3ab83

addaleax · 2017-03-22T04:23:57Z

src/node_api.cc

+  CHECK_ARG(ref);
+
+  v8impl::Reference* reference = reinterpret_cast<v8impl::Reference*>(ref);
+  delete reference;


(re: my comment above … maybe also make the Reference destructor private and add a Delete static? I know it might seem a bit superfluous but I think it would really help with readability)

Resolved by fb8ced4

addaleax · 2017-03-22T04:26:03Z

src/node_api.cc

+}
+
+napi_status napi_escape_handle(napi_env e,
+


v8::Handle → v8::Local (also something for search & replace… 😄)

Resolved by 116bd19

addaleax · 2017-03-22T04:29:23Z

src/node_api.cc

+}
+
+napi_status napi_escape_handle(napi_env e,
+


If we want to support an instanceof implementation that doesn’t conform to ES6 (in that it isn’t based on [Symbol.hasInstance]()), could/should we call it something else? Alternatively, could this wrapper just manually call out to constructor[Symbol.hasInstance](object)?

This does conform to ES6 instanceof. There are extensive tests at test/addons-napi/test_instanceof that re-run all of V8's instanceof tests using this API.

@addaleax I implemented napi_instanceof() this way because I didn't have access to the implementation of the operator. However, if v8 provides access to an implementation that does not require retaining/acquiring the FunctionTemplate then I'd be happy to change this API. Additionally, it looks like Symbol.hasInstance() allows users to customize the behaviour of the instanceof operator, so we absolutely have to use Symbol.hasInstance() when available.

@jasongin Yeah, it matches OrdinaryHasInstance(). I’m good with just naming it that.

Otoh, it should really not be a problem for V8 to expose InstanceOf() and OrdinaryHasInstance() directly…

Re nodejs/node#11975 (comment)

Re nodejs/node#11975 (comment) Fixes nodejs#182 Closes nodejs#185

This updates the documentation, the error message upon module load failure, the command line option parsing of the flag, and the way the N-API addon tests pass the flag to node. Re nodejs/node#11975 (comment) Re nodejs/node#11975 (comment) Fixes nodejs#184

This updates the documentation, the error message upon module load failure, the command line option parsing of the flag, and the way the N-API addon tests pass the flag to node. Re nodejs/node#11975 (comment) Re nodejs/node#11975 (comment) Fixes nodejs#184 Closes nodejs#186

This updates the documentation, the error message upon module load failure, the command line option parsing of the flag, and the way the N-API addon tests pass the flag to node. Re nodejs/node#11975 (comment) Re nodejs/node#11975 (comment) Fixes #184 Closes #186

Re nodejs/node#11975 (comment) Fixes nodejs#182 Closes nodejs#185

This updates the documentation, the error message upon module load failure, the command line option parsing of the flag, and the way the N-API addon tests pass the flag to node. Re nodejs#11975 (comment) Re nodejs#11975 (comment) Fixes nodejs/abi-stable-node#184 Closes nodejs/abi-stable-node#186

addaleax · 2017-04-03T08:36:38Z

Landed in 56e881d 🎉

@nodejs/abi-stable-node I think that also means that from now on you can open PRs against this repository instead of nodejs/abi-stable-node. :)

Add support for abi stable module API (N-API) as "Experimental feature". The goal of this API is to provide a stable Node API for native module developers. N-API aims to provide ABI compatibility guarantees across different Node versions and also across different Node VMs - allowing N-API enabled native modules to just work across different versions and flavors of Node.js without recompilation. A more detailed introduction is provided in: https://github.com/nodejs/node-eps/blob/master/005-ABI-Stable-Module-API.md and https://github.com/nodejs/abi-stable-node/blob/doc/VM%20Summit.pdf. The feature, during its experimental state, will be guarded by a runtime flag "--napi-modules". Only when this flag is added to the command line will N-API modules along with regular non N-API modules be supported. The API is defined by the methods in "src/node_api.h" and "src/node_api_types.h". This is the best starting point to review the API surface. More documentation will follow. In addition to the implementation of the API using V8, which is included in this PR, the API has also been validated against chakracore and that port is available in https://github.com/nodejs/abi-stable-node/tree/api-prototype-chakracore-8.x. The current plan is to provide N-API support in versions 8.X and 6.X directly. For older versions, such as 4.X or pre N-API versions of 6.X, we plan to create an external npm module to provide a migration path that will allow modules targeting older Node.js versions to use the API, albeit without getting the advantage of not having to recompile. In addition, we also plan an external npm package with C++ sugar to simplify the use of the API. The sugar will be in-line only and will only use the exported N-API methods but is not part of the N-API itself. The current version is in: https://github.com/nodejs/node-api. This PR is a result of work in the abi-stable-node repo: https://github.com/nodejs/abi-stable-node/tree/doc, with this PR being the cumulative work on the api-prototype-8.x branch with the following contributors in alphabetical order: Author: Arunesh Chandra <arunesh.chandra@microsoft.com> Author: Gabriel Schulhof <gabriel.schulhof@intel.com> Author: Hitesh Kanwathirtha <hiteshk@microsoft.com> Author: Ian Halliday <ianhall@microsoft.com> Author: Jason Ginchereau <jasongin@microsoft.com> Author: Michael Dawson <michael_dawson@ca.ibm.com> Author: Sampson Gao <sampsong@ca.ibm.com> Author: Taylor Woll <taylor.woll@microsoft.com> PR-URL: #11975 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com>

* **Async Hooks** * The `async_hooks` module has landed in core [[`4a7233c178`](nodejs@4a7233c178)] [nodejs#12892](nodejs#12892). * **Buffer** * Using the `--pending-deprecation` flag will cause Node.js to emit a deprecation warning when using `new Buffer(num)` or `Buffer(num)`. [[`d2d32ea5a2`](nodejs@d2d32ea5a2)] [nodejs#11968](nodejs#11968). * `new Buffer(num)` and `Buffer(num)` will zero-fill new `Buffer` instances [[`7eb1b4658e`](nodejs@7eb1b4658e)] [nodejs#12141](nodejs#12141). * Many `Buffer` methods now accept `Uint8Array` as input [[`beca3244e2`](nodejs@beca3244e2)] [nodejs#10236](nodejs#10236). * **Child Process** * Argument and kill signal validations have been improved [[`97a77288ce`](nodejs@97a77288ce)] [nodejs#12348](nodejs#12348), [[`d75fdd96aa`](nodejs@d75fdd96aa)] [nodejs#10423](nodejs#10423). * Child Process methods accept `Uint8Array` as input [[`627ecee9ed`](nodejs@627ecee9ed)] [nodejs#10653](nodejs#10653). * **Console** * Error events emitted when using `console` methods are now supressed. [[`f18e08d820`](nodejs@f18e08d820)] [nodejs#9744](nodejs#9744). * **Dependencies** * The npm client has been updated to 5.0.0 [[`3c3b36af0f`](nodejs@3c3b36af0f)] [nodejs#12936](nodejs#12936). * V8 has been updated to 5.8 with forward ABI stability to 6.0 [[`60d1aac8d2`](nodejs@60d1aac8d2)] [nodejs#12784](nodejs#12784). * **Domains** * Native `Promise` instances are now `Domain` aware [[`84dabe8373`](nodejs@84dabe8373)] [nodejs#12489](nodejs#12489). * **Errors** * We have started assigning static error codes to errors generated by Node.js. This has been done through multiple commits and is still a work in progress. * **File System** * The utility class `fs.SyncWriteStream` has been deprecated [[`7a55e34ef4`](nodejs@7a55e34ef4)] [nodejs#10467](nodejs#10467). * The deprecated `fs.read()` string interface has been removed [[`3c2a9361ff`](nodejs@3c2a9361ff)] [nodejs#9683](nodejs#9683). * **HTTP** * Improved support for userland implemented Agents [[`90403dd1d0`](nodejs@90403dd1d0)] [nodejs#11567](nodejs#11567). * Outgoing Cookie headers are concatenated into a single string [[`d3480776c7`](nodejs@d3480776c7)] [nodejs#11259](nodejs#11259). * The `httpResponse.writeHeader()` method has been deprecated [[`fb71ba4921`](nodejs@fb71ba4921)] [nodejs#11355](nodejs#11355). * New methods for accessing HTTP headers have been added to `OutgoingMessage` [[`3e6f1032a4`](nodejs@3e6f1032a4)] [nodejs#10805](nodejs#10805). * **Lib** * All deprecation messages have been assigned static identifiers [[`5de3cf099c`](nodejs@5de3cf099c)] [nodejs#10116](nodejs#10116). * The legacy `linkedlist` module has been removed [[`84a23391f6`](nodejs@84a23391f6)] [nodejs#12113](nodejs#12113). * **N-API** * Experimental support for the new N-API API has been added [[`56e881d0b0`](nodejs@56e881d0b0)] [nodejs#11975](nodejs#11975). * **Process** * Process warning output can be redirected to a file using the `--redirect-warnings` command-line argument [[`03e89b3ff2`](nodejs@03e89b3ff2)] [nodejs#10116](nodejs#10116). * Process warnings may now include additional detail [[`dd20e68b0f`](nodejs@dd20e68b0f)] [nodejs#12725](nodejs#12725). * **REPL** * REPL magic mode has been deprecated [[`3f27f02da0`](nodejs@3f27f02da0)] [nodejs#11599](nodejs#11599). * **Src** * `NODE_MODULE_VERSION` has been updated to 57 (nodejs@ec7cbaf266)] [nodejs#12995](nodejs#12995). * Add `--pending-deprecation` command-line argument and `NODE_PENDING_DEPRECATION` environment variable [[`a16b570f8c`](nodejs@a16b570f8c)] [nodejs#11968](nodejs#11968). * The `--debug` command-line argument has been deprecated. Note that using `--debug` will enable the *new* Inspector-based debug protocol as the legacy Debugger protocol previously used by Node.js has been removed. [[`010f864426`](nodejs@010f864426)] [nodejs#12949](nodejs#12949). * Throw when the `-c` and `-e` command-line arguments are used at the same time [[`a5f91ab230`](nodejs@a5f91ab230)] [nodejs#11689](nodejs#11689). * Throw when the `--use-bundled-ca` and `--use-openssl-ca` command-line arguments are used at the same time. [[`8a7db9d4b5`](nodejs@8a7db9d4b5)] [nodejs#12087](nodejs#12087). * **Stream** * `Stream` now supports `destroy()` and `_destroy()` APIs [[`b6e1d22fa6`](nodejs@b6e1d22fa6)] [nodejs#12925](nodejs#12925). * `Stream` now supports the `_final()` API [[`07c7f198db`](nodejs@07c7f198db)] [nodejs#12828](nodejs#12828). * **TLS** * The `rejectUnauthorized` option now defaults to `true` [[`348cc80a3c`](nodejs@348cc80a3c)] [nodejs#5923](nodejs#5923). * The `tls.createSecurePair()` API now emits a runtime deprecation [[`a2ae08999b`](nodejs@a2ae08999b)] [nodejs#11349](nodejs#11349). * A runtime deprecation will now be emitted when `dhparam` is less than 2048 bits [[`d523eb9c40`](nodejs@d523eb9c40)] [nodejs#11447](nodejs#11447). * **URL** * The WHATWG URL implementation is now a fully-supported Node.js API [[`d080ead0f9`](nodejs@d080ead0f9)] [nodejs#12710](nodejs#12710). * **Util** * `Symbol` keys are now displayed by default when using `util.inspect()` [[`5bfd13b81e`](nodejs@5bfd13b81e)] [nodejs#9726](nodejs#9726). * `toJSON` errors will be thrown when formatting `%j` [[`455e6f1dd8`](nodejs@455e6f1dd8)] [nodejs#11708](nodejs#11708). * Convert `inspect.styles` and `inspect.colors` to prototype-less objects [[`aab0d202f8`](nodejs@aab0d202f8)] [nodejs#11624](nodejs#11624). * The new `util.promisify()` API has been added [[`99da8e8e02`](nodejs@99da8e8e02)] [nodejs#12442](nodejs#12442). * **Zlib** * Support `Uint8Array` in Zlib convenience methods [[`91383e47fd`](nodejs@91383e47fd)] [nodejs#12001](nodejs#12001). * Zlib errors now use `RangeError` and `TypeError` consistently [[`b514bd231e`](nodejs@b514bd231e)] [nodejs#11391](nodejs#11391).

* **Async Hooks** * The `async_hooks` module has landed in core [[`4a7233c178`](4a7233c178)] [#12892](#12892). * **Buffer** * Using the `--pending-deprecation` flag will cause Node.js to emit a deprecation warning when using `new Buffer(num)` or `Buffer(num)`. [[`d2d32ea5a2`](d2d32ea5a2)] [#11968](#11968). * `new Buffer(num)` and `Buffer(num)` will zero-fill new `Buffer` instances [[`7eb1b4658e`](7eb1b4658e)] [#12141](#12141). * Many `Buffer` methods now accept `Uint8Array` as input [[`beca3244e2`](beca3244e2)] [#10236](#10236). * **Child Process** * Argument and kill signal validations have been improved [[`97a77288ce`](97a77288ce)] [#12348](#12348), [[`d75fdd96aa`](d75fdd96aa)] [#10423](#10423). * Child Process methods accept `Uint8Array` as input [[`627ecee9ed`](627ecee9ed)] [#10653](#10653). * **Console** * Error events emitted when using `console` methods are now supressed. [[`f18e08d820`](f18e08d820)] [#9744](#9744). * **Dependencies** * The npm client has been updated to 5.0.0 [[`3c3b36af0f`](3c3b36af0f)] [#12936](#12936). * V8 has been updated to 5.8 with forward ABI stability to 6.0 [[`60d1aac8d2`](60d1aac8d2)] [#12784](#12784). * **Domains** * Native `Promise` instances are now `Domain` aware [[`84dabe8373`](84dabe8373)] [#12489](#12489). * **Errors** * We have started assigning static error codes to errors generated by Node.js. This has been done through multiple commits and is still a work in progress. * **File System** * The utility class `fs.SyncWriteStream` has been deprecated [[`7a55e34ef4`](7a55e34ef4)] [#10467](#10467). * The deprecated `fs.read()` string interface has been removed [[`3c2a9361ff`](3c2a9361ff)] [#9683](#9683). * **HTTP** * Improved support for userland implemented Agents [[`90403dd1d0`](90403dd1d0)] [#11567](#11567). * Outgoing Cookie headers are concatenated into a single string [[`d3480776c7`](d3480776c7)] [#11259](#11259). * The `httpResponse.writeHeader()` method has been deprecated [[`fb71ba4921`](fb71ba4921)] [#11355](#11355). * New methods for accessing HTTP headers have been added to `OutgoingMessage` [[`3e6f1032a4`](3e6f1032a4)] [#10805](#10805). * **Lib** * All deprecation messages have been assigned static identifiers [[`5de3cf099c`](5de3cf099c)] [#10116](#10116). * The legacy `linkedlist` module has been removed [[`84a23391f6`](84a23391f6)] [#12113](#12113). * **N-API** * Experimental support for the new N-API API has been added [[`56e881d0b0`](56e881d0b0)] [#11975](#11975). * **Process** * Process warning output can be redirected to a file using the `--redirect-warnings` command-line argument [[`03e89b3ff2`](03e89b3ff2)] [#10116](#10116). * Process warnings may now include additional detail [[`dd20e68b0f`](dd20e68b0f)] [#12725](#12725). * **REPL** * REPL magic mode has been deprecated [[`3f27f02da0`](3f27f02da0)] [#11599](#11599). * **Src** * `NODE_MODULE_VERSION` has been updated to 57 (ec7cbaf266)] [#12995](#12995). * Add `--pending-deprecation` command-line argument and `NODE_PENDING_DEPRECATION` environment variable [[`a16b570f8c`](a16b570f8c)] [#11968](#11968). * The `--debug` command-line argument has been deprecated. Note that using `--debug` will enable the *new* Inspector-based debug protocol as the legacy Debugger protocol previously used by Node.js has been removed. [[`010f864426`](010f864426)] [#12949](#12949). * Throw when the `-c` and `-e` command-line arguments are used at the same time [[`a5f91ab230`](a5f91ab230)] [#11689](#11689). * Throw when the `--use-bundled-ca` and `--use-openssl-ca` command-line arguments are used at the same time. [[`8a7db9d4b5`](8a7db9d4b5)] [#12087](#12087). * **Stream** * `Stream` now supports `destroy()` and `_destroy()` APIs [[`b6e1d22fa6`](b6e1d22fa6)] [#12925](#12925). * `Stream` now supports the `_final()` API [[`07c7f198db`](07c7f198db)] [#12828](#12828). * **TLS** * The `rejectUnauthorized` option now defaults to `true` [[`348cc80a3c`](348cc80a3c)] [#5923](#5923). * The `tls.createSecurePair()` API now emits a runtime deprecation [[`a2ae08999b`](a2ae08999b)] [#11349](#11349). * A runtime deprecation will now be emitted when `dhparam` is less than 2048 bits [[`d523eb9c40`](d523eb9c40)] [#11447](#11447). * **URL** * The WHATWG URL implementation is now a fully-supported Node.js API [[`d080ead0f9`](d080ead0f9)] [#12710](#12710). * **Util** * `Symbol` keys are now displayed by default when using `util.inspect()` [[`5bfd13b81e`](5bfd13b81e)] [#9726](#9726). * `toJSON` errors will be thrown when formatting `%j` [[`455e6f1dd8`](455e6f1dd8)] [#11708](#11708). * Convert `inspect.styles` and `inspect.colors` to prototype-less objects [[`aab0d202f8`](aab0d202f8)] [#11624](#11624). * The new `util.promisify()` API has been added [[`99da8e8e02`](99da8e8e02)] [#12442](#12442). * **Zlib** * Support `Uint8Array` in Zlib convenience methods [[`91383e47fd`](91383e47fd)] [#12001](#12001). * Zlib errors now use `RangeError` and `TypeError` consistently [[`b514bd231e`](b514bd231e)] [#11391](#11391).

Add support for abi stable module API (N-API) as "Experimental feature". The goal of this API is to provide a stable Node API for native module developers. N-API aims to provide ABI compatibility guarantees across different Node versions and also across different Node VMs - allowing N-API enabled native modules to just work across different versions and flavors of Node.js without recompilation. A more detailed introduction is provided in: https://github.com/nodejs/node-eps/blob/master/005-ABI-Stable-Module-API.md and https://github.com/nodejs/abi-stable-node/blob/doc/VM%20Summit.pdf. The feature, during its experimental state, will be guarded by a runtime flag "--napi-modules". Only when this flag is added to the command line will N-API modules along with regular non N-API modules be supported. The API is defined by the methods in "src/node_api.h" and "src/node_api_types.h". This is the best starting point to review the API surface. More documentation will follow. In addition to the implementation of the API using V8, which is included in this PR, the API has also been validated against chakracore and that port is available in https://github.com/nodejs/abi-stable-node/tree/api-prototype-chakracore-8.x. The current plan is to provide N-API support in versions 8.X and 6.X directly. For older versions, such as 4.X or pre N-API versions of 6.X, we plan to create an external npm module to provide a migration path that will allow modules targeting older Node.js versions to use the API, albeit without getting the advantage of not having to recompile. In addition, we also plan an external npm package with C++ sugar to simplify the use of the API. The sugar will be in-line only and will only use the exported N-API methods but is not part of the N-API itself. The current version is in: https://github.com/nodejs/node-api. This PR is a result of work in the abi-stable-node repo: https://github.com/nodejs/abi-stable-node/tree/doc, with this PR being the cumulative work on the api-prototype-8.x branch with the following contributors in alphabetical order: Author: Arunesh Chandra <arunesh.chandra@microsoft.com> Author: Gabriel Schulhof <gabriel.schulhof@intel.com> Author: Hitesh Kanwathirtha <hiteshk@microsoft.com> Author: Ian Halliday <ianhall@microsoft.com> Author: Jason Ginchereau <jasongin@microsoft.com> Author: Michael Dawson <michael_dawson@ca.ibm.com> Author: Sampson Gao <sampsong@ca.ibm.com> Author: Taylor Woll <taylor.woll@microsoft.com> PR-URL: nodejs#11975 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com>

Add support for abi stable module API (N-API) as "Experimental feature". The goal of this API is to provide a stable Node API for native module developers. N-API aims to provide ABI compatibility guarantees across different Node versions and also across different Node VMs - allowing N-API enabled native modules to just work across different versions and flavors of Node.js without recompilation. A more detailed introduction is provided in: https://github.com/nodejs/node-eps/blob/master/005-ABI-Stable-Module-API.md and https://github.com/nodejs/abi-stable-node/blob/doc/VM%20Summit.pdf. The feature, during its experimental state, will be guarded by a runtime flag "--napi-modules". Only when this flag is added to the command line will N-API modules along with regular non N-API modules be supported. The API is defined by the methods in "src/node_api.h" and "src/node_api_types.h". This is the best starting point to review the API surface. More documentation will follow. In addition to the implementation of the API using V8, which is included in this PR, the API has also been validated against chakracore and that port is available in https://github.com/nodejs/abi-stable-node/tree/api-prototype-chakracore-8.x. The current plan is to provide N-API support in versions 8.X and 6.X directly. For older versions, such as 4.X or pre N-API versions of 6.X, we plan to create an external npm module to provide a migration path that will allow modules targeting older Node.js versions to use the API, albeit without getting the advantage of not having to recompile. In addition, we also plan an external npm package with C++ sugar to simplify the use of the API. The sugar will be in-line only and will only use the exported N-API methods but is not part of the N-API itself. The current version is in: https://github.com/nodejs/node-api. This PR is a result of work in the abi-stable-node repo: https://github.com/nodejs/abi-stable-node/tree/doc, with this PR being the cumulative work on the api-prototype-8.x branch with the following contributors in alphabetical order: Author: Arunesh Chandra <arunesh.chandra@microsoft.com> Author: Gabriel Schulhof <gabriel.schulhof@intel.com> Author: Hitesh Kanwathirtha <hiteshk@microsoft.com> Author: Ian Halliday <ianhall@microsoft.com> Author: Jason Ginchereau <jasongin@microsoft.com> Author: Michael Dawson <michael_dawson@ca.ibm.com> Author: Sampson Gao <sampsong@ca.ibm.com> Author: Taylor Woll <taylor.woll@microsoft.com> Backport-PR-URL: #19447 PR-URL: #11975 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com>

nodejs-github-bot added build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++. labels Mar 21, 2017

richardlau reviewed Mar 21, 2017

View reviewed changes

mscdex added the addons Issues and PRs related to native addons. label Mar 21, 2017

mhdawson self-assigned this Mar 21, 2017

trevnorris reviewed Mar 21, 2017

View reviewed changes

mhdawson added the ctc-review label Mar 21, 2017

mhdawson mentioned this pull request Mar 21, 2017

Node.js Foundation Core Technical Committee (CTC) Meeting 2017-03-22 nodejs/CTC#82

Closed

Fix undefined snprintf use for older compiler

a5d43d1

The snprintf API is not available with older versions of MSVC. Found by a CI run.

addaleax added the module Issues and PRs related to the module subsystem. label Mar 22, 2017

addaleax reviewed Mar 22, 2017

View reviewed changes

gabrielschulhof pushed a commit to gabrielschulhof/abi-stable-node that referenced this pull request Mar 22, 2017

doc: update to reflect new command line switch

b97d518

Re nodejs/node#11975 (comment)

gabrielschulhof mentioned this pull request Mar 22, 2017

doc: update to reflect new command line switch nodejs/abi-stable-node#180

Closed

jasnell added this to the 8.0.0 milestone Mar 22, 2017

This was referenced Mar 22, 2017

Use Symbol.hasInstance() for napi_instanceof on V8, when available nodejs/abi-stable-node#182

Closed

Remove [=yes|no] from runtime flag nodejs/abi-stable-node#184

Closed

gabrielschulhof pushed a commit to gabrielschulhof/abi-stable-node that referenced this pull request Mar 22, 2017

napi: change napi_instanceof() to use Symbol.hasInstance

ccee0af

Re nodejs/node#11975 (comment) Fixes nodejs#182 Closes nodejs#185

gabrielschulhof mentioned this pull request Mar 22, 2017

all: update --napi-modules flag to not have a yes/no nodejs/abi-stable-node#186

Closed

This was referenced Mar 22, 2017

Address some minor PR feedback nodejs/abi-stable-node#187

Merged

Use ref/unref terminology instead of addref/release nodejs/abi-stable-node#188

Closed

jasongin mentioned this pull request Mar 22, 2017

Type-checking when getting values nodejs/abi-stable-node#191

Closed

mhdawson added the semver-minor PRs that contain new features and should be released in the next minor version. label Mar 22, 2017

gabrielschulhof pushed a commit to gabrielschulhof/abi-stable-node that referenced this pull request Mar 23, 2017

napi: change napi_instanceof() to use Symbol.hasInstance

bf6fa74

Re nodejs/node#11975 (comment) Fixes nodejs#182 Closes nodejs#185

boingoing mentioned this pull request Mar 23, 2017

Convert all locals and parameter names to snake_case nodejs/abi-stable-node#193

Merged

Trott mentioned this pull request Apr 3, 2017

Node.js Foundation Core Technical Committee (CTC) Meeting 2017-04-05 nodejs/CTC#93

Closed

addaleax closed this Apr 3, 2017

addaleax removed the ctc-review label Apr 3, 2017

mykmelez mentioned this pull request Apr 4, 2017

Merge node mozilla/spidernode#372

Closed

jasnell mentioned this pull request Apr 4, 2017

8.0.0 Release Proposal #12220

Closed

RReverser mentioned this pull request Apr 5, 2017

New Node.js NAPI neon-bindings/neon#202

Closed

addaleax mentioned this pull request Apr 6, 2017

n-api: implement async helper methods #12250

Closed

3 tasks

jasnell mentioned this pull request Apr 25, 2017

Collaborator nominations #12646

Closed

addaleax mentioned this pull request Apr 29, 2017

doc: adding "rules of thumb" #12744

Closed

2 tasks

jasongin mentioned this pull request May 8, 2017

Merge N-API JSRT implementation from abi-stable-node nodejs/node-chakracore#238

Merged

3 tasks

bnoordhuis mentioned this pull request Mar 16, 2018

Adding Websocket support to core #19308

Open

MylesBorins mentioned this pull request Apr 16, 2018

v6.14.2 proposal #19996

Merged

bnoordhuis mentioned this pull request Apr 18, 2018

src: kickstart addon by calling its constructor #20114

Closed

4 tasks

vmoroz mentioned this pull request Jan 25, 2023

node-api: deprecate napi_module_register based registration #46319

Closed

module: add support for abi stable module API #11975

module: add support for abi stable module API #11975

Conversation

jasongin commented Mar 21, 2017 • edited by fhinkel

Checklist

Affected core subsystem(s)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trevnorris left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

addaleax commented Mar 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

addaleax commented Apr 3, 2017

jasongin commented Mar 21, 2017 •

edited by fhinkel