Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

src: handle no support for external buffers #1273

Closed
wants to merge 1 commit into from

Conversation

legendecas
Copy link
Member

@legendecas legendecas commented Jan 16, 2023

Fixes #1257.

  • When NODE_API_NO_EXTERNAL_BUFFERS_ALLOWED introduced in node-api: handle no support for external buffers node#45181 is defined, hides the methods that can create external buffers.
  • Introduce Napi::Buffer::NewOrCopy to create buffer from external buffers that are compatible with runtimes like electron conveniently.

@lovell
Copy link
Contributor

lovell commented Jan 17, 2023

Thank you for adding this feature. It looks like the approach in this PR can (conditionally) remove the presence of Buffer<T>::New() from the API at compile time. Is this intentional? If so, please can this be considered a breaking, semver major change.

@legendecas
Copy link
Member Author

@lovell it is intentional and NODE_API_NO_EXTERNAL_BUFFERS_ALLOWED is following the definition in nodejs/node#45181. I don't think it is a breaking change, since user codes will not be affected if the newly introduced macro NODE_API_NO_EXTERNAL_BUFFERS_ALLOWED is not defined.

### NewOrCopy

Wraps the provided external data into a new `Napi::Buffer` object. When the
[external buffer][] is not supported, allocates a new `Napi::Buffer` object and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[external buffer][] is not supported, allocates a new `Napi::Buffer` object and
[external buffer][] is not supported by the underlying runtime, allocates a new `Napi::Buffer` object and

Copy link
Member

@mhdawson mhdawson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than the suggested wording change to the docs

@mhdawson
Copy link
Member

@legendecas looks like compilation failed on windows:

  buffer.cc
D:\a\node-addon-api\node-addon-api\napi-inl.h(2504,17): error C2065: 'napi_no_external_buffers_allowed': undeclared identifier [D:\a\node-addon-api\node-addon-api\test\build\binding_noexcept_maybe.vcxproj]
D:\a\node-addon-api\node-addon-api\napi-inl.h(2500): message : while compiling class template member function 'Napi::Buffer<uint16_t> Napi::Buffer<uint16_t>::NewOrCopy(napi_env,T *,size_t)' [D:\a\node-addon-api\node-addon-api\test\build\binding_noexcept_maybe.vcxproj]
          with
          [
              T=uint16_t
          ]
D:\a\node-addon-api\node-addon-api\test\buffer.cc(153): message : see reference to function template instantiation 'Napi::Buffer<uint16_t> Napi::Buffer<uint16_t>::NewOrCopy(napi_env,T *,size_t)' being compiled [D:\a\node-addon-api\node-addon-api\test\build\binding_noexcept_maybe.vcxproj]
          with
          [
              T=uint16_t
          ]
D:\a\node-addon-api\node-addon-api\test\buffer.cc(29): message : see reference to class template instantiation 'Napi::Buffer<uint16_t>' being compiled [D:\a\node-addon-api\node-addon-api\test\build\binding_noexcept_maybe.vcxproj]
D:\a\node-addon-api\node-addon-api\napi-inl.h(2529,17): error C2065: 'napi_no_external_buffers_allowed': undeclared identifier [D:\a\node-addon-api\node-addon-api\test\build\binding_noexcept_maybe.vcxproj]
D:\a\node-addon-api\node-addon-api\test\buffer.cc(180): message : see reference to function template instantiation 'Napi::Buffer<uint16_t> Napi::Buffer<uint16_t>::NewOrCopy<`anonymous-namespace'::CreateOrCopyExternalBufferWithFinalize::<lambda_ed832a77ce45243ca6d5fe088f290640>>(napi_env,T *,size_t,Finalizer)' being compiled [D:\a\node-addon-api\node-addon-api\test\build\binding_noexcept_maybe.vcxproj]
          with
          [
              T=uint16_t,
              Finalizer=`anonymous-namespace'::CreateOrCopyExternalBufferWithFinali

@Julusian
Copy link

It looks like when NODE_API_NO_EXTERNAL_BUFFERS_ALLOWED is set, that Buffer::NewOrCopy is being defined, with the implementation attempting to use napi_create_external_buffer which is missing because of the NODE_API_NO_EXTERNAL_BUFFERS_ALLOWED flag.

When NODE_API_NO_EXTERNAL_BUFFERS_ALLOWED either Buffer::NewOrCopy should also be disabled, or the implementation should skip calling napi_create_external_buffer and always do the copy?

Ideally I would like to be able to use Buffer::NewOrCopy which is able to do napi_create_external_buffer, but to also have the offending Buffer::New methods removed. That way I can't accidentally use Buffer::New, but I can still get the benefits of Buffer::NewOrCopy when the runtime allows. I don't know how this could be implemented though

@legendecas
Copy link
Member Author

@Julusian thanks for the suggestion! I'll update the PR to include a test to verify Buffer::NewOrCopy is still usable when NODE_API_NO_EXTERNAL_BUFFERS_ALLOWED is defined.

@legendecas
Copy link
Member Author

legendecas commented Feb 8, 2023

Seems like the test async_progress_worker is constantly failing on windows-2019, v16.x: #1278 (comment)


template <typename T>
template <typename Finalizer, typename Hint>
inline Buffer<T> Buffer<T>::NewOrCopy(napi_env env,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume its the templates that prevent us from having one function with all the parameters and the others calling that one with dummy/null parameters, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. When the template parameter Hint is set, the Finalizer must accept a third parameter Hint too. However, we allow Finalizer to be like void operator()(Env env, T* data). So we have to distinguish if Hint is present.

Copy link
Member

@mhdawson mhdawson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM on updated version with one question

@legendecas
Copy link
Member Author

@mhdawson is this still looking good to you with the replied question?

If so, I think we've discussed about the failure of Node.js CI Windows Platform / test (16.x, x86, windows-2019) (pull_request) and we can suppress it at the moment and land this PR.

mhdawson pushed a commit that referenced this pull request Feb 21, 2023
PR-URL: #1273
Reviewed-By: Michael Dawson <midawson@redhat.com
@mhdawson
Copy link
Member

@legendecas LGTM
Landed as b16c762

@mhdawson mhdawson closed this Feb 21, 2023
@legendecas legendecas deleted the buffer branch February 22, 2023 02:01
austinli64 added a commit to austinli64/node-addon-api that referenced this pull request May 9, 2023
PR-URL: nodejs/node-addon-api#1273
Reviewed-By: Michael Dawson <midawson@redhat.com
johnfrench3 pushed a commit to johnfrench3/node-addon-api-git that referenced this pull request Aug 11, 2023
PR-URL: nodejs/node-addon-api#1273
Reviewed-By: Michael Dawson <midawson@redhat.com
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide a new Buffer::NewOrCopy method
4 participants