n-api: fix warning about size_t compare with int #15508

sampsongao · 2017-09-20T21:42:13Z

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

mhdawson

LGTM

mhdawson · 2017-09-20T21:54:02Z

CI Run: https://ci.nodejs.org/job/node-test-pull-request/10171/

fhinkel · 2017-09-21T01:53:12Z

src/node_api.cc

@@ -920,13 +920,13 @@ NAPI_NO_RETURN void napi_fatal_error(const char* location,
                                     size_t message_len) {
  char* location_string = const_cast<char*>(location);
  char* message_string = const_cast<char*>(message);
-  if (location_len != -1) {
+  if (static_cast<int>(location_len) != -1) {


size_t might allow for larger values than signed ints and we're wrongly casting 2^n-1 to -1 (where n is how big size_t is). If we want to allow -1 here, why don't we change the parameter type? Or could we use 0 instead of -1 to signal that we don't want to set the message_string?

bnoordhuis · 2017-09-21T10:09:53Z

src/node_api.cc

@@ -920,13 +920,13 @@ NAPI_NO_RETURN void napi_fatal_error(const char* location,
                                     size_t message_len) {
  char* location_string = const_cast<char*>(location);
  char* message_string = const_cast<char*>(message);
-  if (location_len != -1) {
+  if (static_cast<int>(location_len) != -1) {
    location_string = reinterpret_cast<char*>(
        malloc(location_len * sizeof(char) + 1));


Aside: sizeof(char) is 1 by definition and reinterpret_cast is overkill here, static_cast would have sufficed.

What's more, I don't understand why this code makes copies. This function does not return. It seems like completely unneeded complexity.

The reason for doing copy here is to account for strings that are not null terminated.

Okay, but the other two points still stand. As well, you don't handle nullptr return values. If you use node::Malloc(), that's taken care of.

Or use std::string, that will also stop Coverity from complaining about memory leaks.

mhdawson · 2017-09-21T13:00:21Z

@sampsongao can you look at the types that are used by the similar calls in V8 that already had the -1 as the option to indicate null terminated ? We were already passing through -1 to those calls and maybe we need to use the same type here.

mhdawson · 2017-09-21T14:27:08Z

Talking with Sampson, in many cases we just pass the -1 on to an v8 call so I don't think we'd want to switch to using 0 instead as then we'd need code to covert.

It may make sense to just change the type to int. I think we discussed size_t versus int before but we may not have factored in that -1 is not valid for size_t

bnoordhuis · 2017-09-21T15:42:56Z

-1 is not valid for size_t

It's valid (it wraps around to SIZE_MAX), it probably just isn't what's intended.

Talking with Sampson, in many cases we just pass the -1 on to an v8 call so I don't think we'd want to switch to using 0 instead as then we'd need code to covert.

What about other engines? Abstracting away such details is the point of n-api, isn't it?

jasongin · 2017-09-21T15:47:38Z

For some APIs that take a string length, for example napi_create_string_utf8(), 0 is a valid length. So I don't think 0 should be used to indicate automatic length.

Should we consider ssize_t (the signed variant of size_t)? While it's not part of the C++ standard, node already uses it in many places.

bnoordhuis · 2017-09-21T16:03:40Z

What's wrong with passing the size explicitly? The only reason I can think of is programmer ergonomics but passing -1 or strlen(s) isn't that much of a difference.

mhdawson · 2017-09-21T17:07:58Z

We'd discussed the options in the past and agreed to support the -1 shortcut in the way that V8 does. The latest PR was just to make it consistent across the board. I think we should fix up the type problem and then discuss if we should remove the shortcut completely in a later PR.

jasongin · 2017-09-21T17:14:27Z

What's wrong with passing the size explicitly?

That would be fine with me also. In most cases the C++ wrapper would make that automatic anyway. But if we do that we should probably still detect when -1 is passed in, because the compiler usually won't complain and string APIs in many libraries do support it, so people may assume it works with N-API.

Or we could document that SIZE_MAX indicates automatic length, and then -1 would ("accidentally") work the same way. Actually I sort of like this, it seems like a good compromise.

mhdawson · 2017-09-21T17:29:13Z

SIZE_MAX would be fine with me as long as that works in the cases were we pass it on to V8 and its sounds like that will be the case.

mhdawson · 2017-09-21T17:48:06Z

We discussed in N-API meeting today. Here is what we'd like to do

Introduce a new constant NAPI_AUTO_LENGTH
Define NAPI_AUTO_LENGTH as SIZE_MAX
Fix up the checks we have that we were getting warning on to check against NAPI_AUTO_LENGTH
Fix up doc to indicate that people should used NAPI_AUTO_LENGTH instead of -1.

mhdawson · 2017-09-21T17:48:47Z

Lets update this PR to do 1-3 and I will submit a separate PR to update the doc.

mhdawson

LGTM

mhdawson · 2017-09-22T13:12:35Z

CI run: https://ci.nodejs.org/job/node-test-pull-request/10192/

@bnoordhuis, @fhinkel can you take a another look, we'd like this to make it into 8.6 which is going out next tuesday.

addaleax · 2017-09-22T13:21:07Z

src/node_api.cc

@@ -120,13 +120,14 @@ struct napi_env__ {
 #define CHECK_NEW_FROM_UTF8_LEN(env, result, str, len)                   \
  do {                                                                   \
    auto str_maybe = v8::String::NewFromUtf8(                            \
-        (env)->isolate, (str), v8::NewStringType::kInternalized, (len)); \
+        (env)->isolate, (str), v8::NewStringType::kInternalized,         \
+        static_cast<int>(len));                                          \


It would be nice to have a static_assert(static_cast<int>(NAPI_AUTO_LENGTH) == -1) somewhere

@addaleax that is a good idea.

Do you think it would make sense to add it just after

namespace {
namespace v8impl {

in node_api.cc

along with a comment indicating the v8 implementation of n-api depends on it being true ?

@mhdawson I like to put that near where it’s actually used (i.e. around here) but feel free to put it wherever you like :)

I was thinking you had somewhere else in mind, I'm good with it going into the macro itself

mhdawson · 2017-09-22T14:52:35Z

centos failures are #15505

mhdawson · 2017-09-22T14:57:41Z

Windows fanned failure was unrelated created - #15558

mhdawson · 2017-09-22T14:58:34Z

Arm failure was an infra issue, not test related.

So overall CI looks good.

fhinkel · 2017-09-22T15:17:16Z

src/node_api.cc

+  std::string location_string;
+  std::string message_string;
+
+  if (static_cast<int>(location_len) != NAPI_AUTO_LENGTH) {


I don't think that size_t always fits into an int. Why do we need to cast here? (Feel free to land once this is addressed if I don't get around to change my review to approve in time.)

I agree it should not be needed in this check. In places where it is passed on to v8 then it is needed.

@mhdawson No, the issue isn’t that it’s in the wrong place – it’s that a downcast is a real bug here, because it truncates location_len

@addaleax, to clarify we know that size_t does not fit into an int, so where its going to be passed to v8 a cast is required to avoid warnings. In this case we should just be comparing NAPI_AUTO_LENGTH directly against location_len.

@mhdawson Yeah, but we can’t just downcast without any checks in this case. Imagine a platform that has 32-bit int, and 64-bit size_t. When an addon tries to create a N-API string that contains (2^32 + 1) characters – not knowing that that doesn’t work, because, how would it tell? – N-API would currently silently truncate that size argument to 1, and happily create a one-character string. I think that qualifies as a bug.

I think I'm still missing something here, sorry. If we change to be:

if (location_len != NAPI_AUTO_LENGTH)

is it that there is a problem later on ?

Make checks directly against NAPI_AUTO_LENGTH Add assert Remove one check in unrelated napi function that was causing a warning and is not needed

mhdawson · 2017-09-22T19:32:08Z

Pushed change to address comments (I think)

mhdawson · 2017-09-22T19:33:14Z

New ci run to make sure assert validates we have the right behavior on all platforms: https://ci.nodejs.org/job/node-test-pull-request/10206/

mhdawson · 2017-09-22T20:51:01Z

Latest CI run was good, only failure is known pre-existing problem on centos

addaleax · 2017-09-22T21:14:54Z

src/node_api.cc

    auto str_maybe = v8::String::NewFromUtf8(                            \
-        (env)->isolate, (str), v8::NewStringType::kInternalized, (len)); \
+        (env)->isolate, (str), v8::NewStringType::kInternalized,         \
+        static_cast<int>(len));                                          \


@mhdawson The issue remains that this static_cast is invalid if len > INT_MAX && len != NAPI_AUTO_LENGTH, there needs to code that checks for that condition and errors out if it is met … if necessary, I guess that can be done in a subsequent PR, but it’s a bug that ought to be fixed.

@addaleax thanks for the clarification, will look at that. Since we want to have the breaking changes in before 8.6 and we are almost out of time I may do it in a follow up PR, but I'll see on Monday.

I think this check may do the trick

RETURN_STATUS_IF_FALSE((env), len <= INT_MAX, (napi_invalid_arg));

I do think you need to check for NAPI_AUTO_LENGTH because that value is usually larger than INT_MAX :) But yes, something like that sounds good

@addaleax agreed I also need to check for NAPI_AUTO_LENGTH. I'm going to land this PR to make sure we don't fix the Tuesday cutoff, and I'll submit a separate one to fix the bug caused by the cast on Monday.

To be clear, the current check does not have well-defined behavior. Compiler A might do the right thing, compiler B might do something completely different. IOW, this is not release-ready code.

PR-URL: #15508 Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

mhdawson · 2017-09-24T14:51:22Z

Landed as a406a32

Will leave open until I open follow on PR on monday.

mhdawson · 2017-09-25T15:43:15Z

PR to add check on length #15611

PR-URL: #15508 Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

PR-URL: nodejs/node#15508 Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

PR-URL: nodejs#15508 Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

Backport-PR-URL: #19447 PR-URL: #15508 Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

n-api: fix warning about size_t compare with int

657d660

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. dont-land-on-v4.x node-api Issues and PRs related to the Node-API. labels Sep 20, 2017

mhdawson approved these changes Sep 20, 2017

View reviewed changes

cjihrig approved these changes Sep 20, 2017

View reviewed changes

jasnell approved these changes Sep 20, 2017

View reviewed changes

fhinkel requested changes Sep 21, 2017

View reviewed changes

bnoordhuis reviewed Sep 21, 2017

View reviewed changes

digitalinfinity mentioned this pull request Sep 21, 2017

n-api: Add optional length to functions accepting const char * #15108

Closed

Sampson Gao added 2 commits September 21, 2017 18:23

Define NAPI_AUTO_LENGTH for string length

b1bf44a

Use std::string for copying string

a7ab686

mhdawson approved these changes Sep 22, 2017

View reviewed changes

addaleax approved these changes Sep 22, 2017

View reviewed changes

fhinkel reviewed Sep 22, 2017

View reviewed changes

squash: address latest comments

c4732e7

Make checks directly against NAPI_AUTO_LENGTH Add assert Remove one check in unrelated napi function that was causing a warning and is not needed

fhinkel approved these changes Sep 22, 2017

View reviewed changes

addaleax reviewed Sep 22, 2017

View reviewed changes

mhdawson mentioned this pull request Sep 25, 2017

Node.js 8.6.0 Release Proposal #15509

Merged

mhdawson closed this Sep 25, 2017

MylesBorins mentioned this pull request Apr 16, 2018

v6.14.2 proposal #19996

Merged

n-api: fix warning about size_t compare with int #15508

n-api: fix warning about size_t compare with int #15508

Conversation

sampsongao commented Sep 20, 2017 • edited by mhdawson

Checklist

Affected core subsystem(s)

mhdawson left a comment

Choose a reason for hiding this comment

mhdawson commented Sep 20, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhdawson commented Sep 21, 2017

mhdawson commented Sep 21, 2017 • edited

bnoordhuis commented Sep 21, 2017

jasongin commented Sep 21, 2017

bnoordhuis commented Sep 21, 2017

mhdawson commented Sep 21, 2017

jasongin commented Sep 21, 2017

mhdawson commented Sep 21, 2017

mhdawson commented Sep 21, 2017

mhdawson commented Sep 21, 2017

mhdawson left a comment

Choose a reason for hiding this comment

mhdawson commented Sep 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhdawson commented Sep 22, 2017

mhdawson commented Sep 22, 2017

mhdawson commented Sep 22, 2017

fhinkel Sep 22, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhdawson commented Sep 22, 2017

mhdawson commented Sep 22, 2017

mhdawson commented Sep 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhdawson Sep 22, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhdawson commented Sep 24, 2017

mhdawson commented Sep 25, 2017

sampsongao commented Sep 20, 2017 •

edited by mhdawson

mhdawson commented Sep 21, 2017 •

edited

fhinkel Sep 22, 2017 •

edited

mhdawson Sep 22, 2017 •

edited