src: add NODE_MODULE_VERSION registry

[rvagg]

Ref: https://nodejs.org/en/download/releases/
Ref: https://github.com/lgeiger/node-abi/blob/master/index.js
Ref: nodejs/TSC#621

As per nodejs/TSC#621, here's a suggestion for how we keep track of these values as they spiral out of control. Already the comments in node_version.h is out of date and the suggested means of "reserving" a value by opening an issue at nodejs/TSC is going to be difficult to track.

I've collected the values as best as I can. Partly from our own history but also @lgeiger's node-abi package and a bit of digging. I couldn't figure out how NW.js stores NODE_MODULE_VERSION so can't update their values (maybe @rogerwang can help?).

Versions are all node-semver range compliant. The V8 increments are pinned to the precise pre-release version they'd compile as if you did it raw (won't match nightly versions unfortunately).

Thoughts?

/cc @nodejs/addon-api @nodejs/v8 @kapouer

[nodejs-github-bot]

@rvagg build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/1501/pipeline

[nicolasnoble]

Having some data structure intersection with https://nodejs.org/dist/index.json might be a good idea in order to make parsing a bit less cumbersome?

[rvagg]

Any suggestions on that @nicolasnoble? index.json has NODE_MODULE_VERSION in it already as "modules", are you suggesting just a rename here?

Cross-referencing the two files would be a matter of filtering by "runtime" and "variant" and then checking semver range compatibility. But since "modules" is already in index.json I'm not sure the utility of doing that. This is more helpful for reservation purposes but could also be easily consumed by prebuild tooling.

[nicolasnoble]

Yes, I'm basically just suggesting a rename so that the same kind of tooling can be used to parse both files :-)

[rvagg]

fair call, not sure it's self documenting enough in here though, it certainly looks more pleasant and compact:

Latest commit has the latter, previous commit has the former. Anyone else have input?

[Flarna]

What I miss here is some more information about where a specific variant is maintained to get the full diff.

Maybe the names used in variant field could be more explicit or include some URL to clearly point to the "owner" of the version number.

[rvagg]

@Flarna I would hope that over time the commit log would begin to show who owns what, if you're reserving a number then you're likely going to have a commit to do it. Do we need to surface additional metadata in the file itself? What would be the use-case for that?

[Flarna]

As a native addon developer I have to decide if I want to support these variants. A change in module version can be any sort of change (not just ABI) and may need also adaptions in other modules like NAN.

I'm not sure if we can expect that variant vendors take care to adapt also NAN so this will be left as exercise for addon developers. To actually do this adaptions it's needed to know all details of the difference.

But not all variants are of interest for everyone.

There are several criterias like e.g.:

• purpose of the variant: electron differs in usecase to node so at lot addons may decide to skip support for it as it is not applicable
• number of users of the variant: e.g. -pre versions of node are hopefully not used in production so skipping them is likely valid for a lot users
• debian users usually don't want closed source/precompiled dependencies ==> no need to maintain a per-compiled binary for the debian variants.

[rogerwang]

Thanks for CC @rvagg . NW.js keeps tracks of the Node version it used in https://nwjs.io/versions.json . I can certainly help update the registry file in Node.

[rvagg]

@Flarna right, so this is mostly about making this more informative for addon authors. I get that, but I'm wondering if that's best achieved in this file or maybe in a matching .md file that can be more freeform. I'll have a play with extending this file and see if I can keep mess to a minimum in the process.

[rvagg]

@rogerwang I might ping you again if/when this lands so you can PR an update. I'm not sure if I can make a 1:1 matching of Node version to NODE_MODULE_VERSION since you're also using -pre versions. So it might be best if you do a separate PR against the file to get those versions recorded properly.

[Flarna]

@rvagg I don't care much about the format here so moving the details into an .md file is also fine - as long as they stay in sync.

[rvagg]

it's the sync problems I'm most concerned about, already node_version.h is out of sync and it's all in one file!

[rvagg]

Updated this to reserve 68 which is in use now for nightlies

  { "modules": 68, "runtime": "node",     "variant": "v8_7.1",               "versions": "12.0.0-pre" },

TODO here is:

• Consider some future-proofing given that we may be adjusting approach in the future, possibly by letting embedders use a uniqie higher-order bit so they can increment independently - we'll want a record of who is using what value in a registry so we can hand those out uniquely, plus a registry of values in use will still be useful for prebuild tools so we'll want to encourage them to report values back to us. We may also have something like POC: src: add addon ABI declaration option #25539 in use and we need to consider if that has any impact on a registry, it may or may not make sense to have some of that information included in here (perhaps just what _ABI_ numbers correspond to what dependencies?)
• Consider an expansion to the "runtime" and/or "variant" that provides more information for addon authors as per @Flarna's comments. I'm thinking that a separate top-level object in here, "variants" or something like that, with at least some descriptive text to explain what it is, who maintains it, a url, maybe even something machine readable to make easy decisions for prebuilding ("release": false?).

I think the obvious step in getting there for now is to just bump this whole structure down one level so more information can be introduced at the top level and this information could potentially be archived if we ever arrived at a full replacement. Something like this:

[rvagg]

added 69 for Electron 4 and 70 for Electron 5 per discussion in nodejs/TSC#651

[rvagg]

This is ready to land (IMO), could I get some reviews please? If you disagree with what's in this PR please make that clear.

Latest changes:

• Updated for V8 7.4 which is expected to be the Node 12 version (72)
• Renamed to doc/abi_version_registry.json
• Turned the JSON structure into an object and embedded the modules version number array into a "NODE_MODULE_VERSION" property

So this is now designed to be extended into the future, it'll support:

• Adding further metadata, perhaps URLs and contact details for owners of different "runtime"s and/or "variant"s (I don't think that's necessary to get this landed and am not convinced it's needed at all but perhaps someone else is)
• New versions of ABI matching that may come from future work, such as the match-per-dependency-version approach that @ofrobots has been outlining in NODE_MODULE_VERSION for Electron Major Releases TSC#651

[nicolasnoble]

Electron 4.0.4 changed its NVM from 64 to 69. There has been debate over if this constituted an ABI breaking change or a fix, and if this was relevant in a patch release or in a major release. See the mentions in electron/electron#16687 and the discussion in electron/node@8bc5d17

It might be worth discussing this here also in order to provide clarification and guidance in the documentation.

[rvagg]

@nicolasnoble it did come up in nodejs/TSC#651 and the reservation of 69 was as a result of that discussion. It's less than ideal for addon authors and consumers in Electron but Electron has additional problems with ABI stability that it might need to monitor more carefully into the future. Having this as a place to register as many new numbers as they need should help resolve concerns about bumping more liberally.

I've made changes to be more specific about Electron versions (5da2462):

-    { "modules": 69, "runtime": "electron", "variant": "electron",             "versions": "4" },
+    { "modules": 69, "runtime": "electron", "variant": "electron",             "versions": "^4.0.5" },

and

-    { "modules": 64, "runtime": "electron", "variant": "electron",             "versions": "3" },
+    { "modules": 64, "runtime": "electron", "variant": "electron",             "versions": ">=3 <4.0.5" },

[ryzokuken]

Thanks, @rvagg!

[nicolasnoble]

The change happened between 4.0.3 and 4.0.4, so your patch here is a little bit off.

[mhdawson]

LGTM once @vsemozhetbyt comments are added.

[murgatroid99]

Can you state in the BUILDING file that changing the NODE_MODULE_VERSION in a Node distribution is an ABI breaking change and should therefore be accompanied by a major version bump because the number determines whether the runtime will load an addon?

In other words, a module built for NODE_MODULE_VERSION X can not be loaded by a Node runtime with NODE_MODULE_VERSION Y, so changing the NODE_MODULE_VERSION in the runtime from X to Y breaks compatibility with installed addons.

[rvagg]

Suggestions all adopted, I think this is ready for landing and I'll do that tomorrow unless there objections.

@murgatroid99 to your point: that's covered fairly comprehensively in doc/releases.md under the heading "Also consider whether to bump NODE_MODULE_VERSION". Beyond the releases we make here the best we can do is help coordinate NODE_MODULE_VERSIONs. What embedders do with their version numbers is out of our hands and how the addon ecosystem handles that is also not something we have much control over—but having a registry of these numbers is certainly going to make it easier!

[rvagg]

Landed in c61c722