Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: Auto-load ICU data from --with-icu-default-data-dir #30825

Closed
wants to merge 1 commit into from

Conversation

sgallagher
Copy link
Contributor

@sgallagher sgallagher commented Dec 6, 2019

When compiled with --with-intl=small and
--with-icu-default-data-dir=PATH, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Related: #3460

Signed-off-by: Stephen Gallagher sgallagh@redhat.com

Attn: @srl295

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++. v12.x labels Dec 6, 2019
Copy link
Member

@srl295 srl295 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor change, but the shape of this looks great to me.

  • It might be helpful to somewhere document the scenario as you did in your comment on the other issue. This could just be in the PR for now.

configure.py Outdated Show resolved Hide resolved
node.gypi Outdated Show resolved Hide resolved
src/node.cc Outdated Show resolved Hide resolved
src/node.cc Show resolved Hide resolved
configure.py Outdated Show resolved Hide resolved
configure.py Outdated Show resolved Hide resolved
src/node.cc Outdated Show resolved Hide resolved
src/node.cc Outdated Show resolved Hide resolved
src/node.cc Outdated Show resolved Hide resolved
src/node.cc Outdated Show resolved Hide resolved
@sgallagher sgallagher changed the title Auto-load ICU data from --with-icu-default-data-dir build: Auto-load ICU data from --with-icu-default-data-dir Dec 9, 2019
@sgallagher
Copy link
Contributor Author

I updated the patch with the code review recommendations. I also changed the commit message and the first comment on this PR to match it and provide more detail on the problem this is solving.

Thank you very much for the review!

@richardlau
Copy link
Member

Shouldn't this target master?

@sgallagher
Copy link
Contributor Author

Rebased it to master. I was working on it against v12 originally because a) that's what we have in Fedora and b) the master branch wasn't compiling successfully for me, but that's been fixed in the interim it seems.

Copy link
Member

@srl295 srl295 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@srl295 srl295 added the i18n-api Issues and PRs related to the i18n implementation. label Dec 10, 2019
@srl295
Copy link
Member

srl295 commented Dec 10, 2019

fyi @nodejs/intl

src/node.cc Outdated Show resolved Hide resolved
src/node.cc Outdated Show resolved Hide resolved
When compiled with `--with-intl=small` and
`--with-icu-default-data-dir=PATH`, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Refs: nodejs#3460

Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>
@sgallagher
Copy link
Contributor Author

Rebased atop the latest master and made the two minor changes @bnoordhuis requested.

@nodejs-github-bot
Copy link
Collaborator

@bnoordhuis bnoordhuis added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. and removed v12.x labels Dec 12, 2019
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@Trott
Copy link
Member

Trott commented Dec 14, 2019

Landed in d502b83

@Trott Trott closed this Dec 14, 2019
Trott pushed a commit that referenced this pull request Dec 14, 2019
When compiled with `--with-intl=small` and
`--with-icu-default-data-dir=PATH`, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Refs: #3460

Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>

PR-URL: #30825
Reviewed-By: Steven R Loomis <srloomis@us.ibm.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
MylesBorins pushed a commit that referenced this pull request Dec 17, 2019
When compiled with `--with-intl=small` and
`--with-icu-default-data-dir=PATH`, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Refs: #3460

Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>

PR-URL: #30825
Reviewed-By: Steven R Loomis <srloomis@us.ibm.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
@MylesBorins MylesBorins mentioned this pull request Dec 17, 2019
targos pushed a commit that referenced this pull request Jan 14, 2020
When compiled with `--with-intl=small` and
`--with-icu-default-data-dir=PATH`, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Refs: #3460

Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>

PR-URL: #30825
Reviewed-By: Steven R Loomis <srloomis@us.ibm.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
BethGriggs pushed a commit that referenced this pull request Feb 6, 2020
When compiled with `--with-intl=small` and
`--with-icu-default-data-dir=PATH`, Node.js will use PATH as a
fallback location for the ICU data.

We will first perform an access check using fopen(PATH, 'r') to
ensure that the file is readable. If it is, we'll set the
icu_data_directory and proceed. There's a slight overhead for the
fopen() check, but it should be barely measurable.

This will be useful for Linux distribution packagers who want to
be able to ship a minimal node binary in a container image but
also be able to add on the full i18n support where needed. With
this patch, it becomes possible to ship the interpreter as
/usr/bin/node in one package for the distribution and to ship the
data files in another package (without a strict dependency
between the two). This means that users of the distribution will
not need to explicitly direct Node.js to locate the ICU data. It
also means that in environments where full internationalization is
not required, they do not need to carry the extra content (with
the associated storage costs).

Refs: #3460

Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>

PR-URL: #30825
Reviewed-By: Steven R Loomis <srloomis@us.ibm.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
@MylesBorins MylesBorins mentioned this pull request Feb 8, 2020
@sgallagher sgallagher deleted the i18n-default-dir branch February 14, 2020 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++. i18n-api Issues and PRs related to the i18n implementation.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants