Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v10.x backport] doc,tools: get altDocs versions from CHANGELOG.md #32642

Closed

Conversation

richardlau
Copy link
Member

@richardlau richardlau commented Apr 3, 2020

Backports of the following PRs to v10.x-staging:

  1. doc,tools: get altDocs versions from CHANGELOG.md #27661
  2. test: fix flaky doctool and test #29979
  3. tools: make doctool work if no internet available #30214
  4. tools: add NODE_TEST_NO_INTERNET to the doc builder #31849
  5. tools: only fetch previous versions when necessary #32518
  6. test: fix tool path in test-doctool-versions.js #32645

The first and fifth of these required manual backporting while the middle three in the sequence cherry-picked cleanly. Does Backport-PR-URL: metadata go on all of the commits in this backport PR or only the ones that didn't cherry-pick cleanly (@nodejs/release)?

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. doc Issues and PRs related to the documentations. tools Issues and PRs related to the tools directory. v10.x labels Apr 3, 2020
@nodejs-github-bot
Copy link
Collaborator

@richardlau
Copy link
Member Author

🤔 Test is failing across the CI. Passes locally for me, and when I log into the CI host and run manually:

[iojs@test-ibm-rhel7-s390x-4 rhel7-s390x]$ ./node --version
v10.19.1-pre
[iojs@test-ibm-rhel7-s390x-4 rhel7-s390x]$ NODE_DEBUG=test ./node test/doctool/test-doctool-versions.js
TEST 62870: /home/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel7-s390x/test/.tmp/versions.json
TEST 62870:
TEST 62870:
TEST 62870: [ { num: '13.x' },
  { num: '12.x', lts: true },
  { num: '11.x' },
  { num: '10.x', lts: true },
  { num: '9.x' },
  { num: '8.x' },
  { num: '7.x' },
  { num: '6.x' },
  { num: '5.x' },
  { num: '4.x' },
  { num: '0.12.x' },
  { num: '0.10.x' } ]
[iojs@test-ibm-rhel7-s390x-4 rhel7-s390x]$

@richardlau
Copy link
Member Author

richardlau commented Apr 3, 2020

Okay it's a bug in the original PR if NODE_TEST_DIR is set to a path outside of the test directory but wasn't spotted because doctool tests are excluded from default in master but not in v10.x-staging. I'll PR a fix to master and backport it into this PR for 10.x.

@richardlau richardlau added the blocked PRs that are blocked by other issues or PRs. label Apr 3, 2020
@richardlau
Copy link
Member Author

Blocked on #32645 landing on master.

richardlau and others added 6 commits April 6, 2020 05:43
Parse `CHANGELOG.md` for versions of Node.js used by the documentation
feature `View another version` so that we don't have to manually update
the list when we cut a new version or transition a release to LTS.

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#27661
Reviewed-By: Rich Trott <rtrott@gmail.com>
Doctool tests have been failing a lot in CI on Win2008 R2. It appears
async functions and callback-based functions are being used in
combination such that the callback-based function cannot guarantee that
it will invoke its callback. Convert the callback-based functions to
async functions so we have one paradigm and reliable results.

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#29979
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Allow doctool to fallback to use local files if not building a release
build.

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#30214
Fixes: nodejs#29918
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
At the moment the doc builder tries to access the internet
for CHANGELOG information and only falls back to local sources
after the connection fails or a 5 second timeout. This means
that the doc building could take at least 7 minutes on a
machine with hijacked connection to Github for useless network
attempts. This patch adds a NODE_TEST_NO_INTERNET environment
variable to directly bypass these attempts so that docs can be
built in reasonable time on a machine like that.

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#31849
Fixes: nodejs#29918
Reviewed-By: Matheus Marchini <mat@mmarchini.me>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Refactor the logic for working out the previous versions of Node.js for
the API documentation so that the parsing (including the potential https
get) happens at most once per build (as opposed to the current once per
generated API doc).

Signed-off-by: Richard Lau <riclau@uk.ibm.com>

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#32518
Fixes: nodejs#32512
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Path to the versions tool tested by test-doctool-versions.js would
be incorrect if the test temporary directory was redirected (e.g.
via NODE_TEST_DIR) outside of `test/`.

Signed-off-by: Richard Lau <riclau@uk.ibm.com>

Backport-PR-URL: nodejs#32642
PR-URL: nodejs#32645
Refs: nodejs#32518
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Shelley Vohr <codebytere@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@richardlau richardlau removed the blocked PRs that are blocked by other issues or PRs. label Apr 6, 2020
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Apr 6, 2020

BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
Parse `CHANGELOG.md` for versions of Node.js used by the documentation
feature `View another version` so that we don't have to manually update
the list when we cut a new version or transition a release to LTS.

Backport-PR-URL: #32642
PR-URL: #27661
Reviewed-By: Rich Trott <rtrott@gmail.com>
BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
Doctool tests have been failing a lot in CI on Win2008 R2. It appears
async functions and callback-based functions are being used in
combination such that the callback-based function cannot guarantee that
it will invoke its callback. Convert the callback-based functions to
async functions so we have one paradigm and reliable results.

Backport-PR-URL: #32642
PR-URL: #29979
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
Allow doctool to fallback to use local files if not building a release
build.

Backport-PR-URL: #32642
PR-URL: #30214
Fixes: #29918
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
At the moment the doc builder tries to access the internet
for CHANGELOG information and only falls back to local sources
after the connection fails or a 5 second timeout. This means
that the doc building could take at least 7 minutes on a
machine with hijacked connection to Github for useless network
attempts. This patch adds a NODE_TEST_NO_INTERNET environment
variable to directly bypass these attempts so that docs can be
built in reasonable time on a machine like that.

Backport-PR-URL: #32642
PR-URL: #31849
Fixes: #29918
Reviewed-By: Matheus Marchini <mat@mmarchini.me>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
Refactor the logic for working out the previous versions of Node.js for
the API documentation so that the parsing (including the potential https
get) happens at most once per build (as opposed to the current once per
generated API doc).

Signed-off-by: Richard Lau <riclau@uk.ibm.com>

Backport-PR-URL: #32642
PR-URL: #32518
Fixes: #32512
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
BethGriggs pushed a commit that referenced this pull request Apr 6, 2020
Path to the versions tool tested by test-doctool-versions.js would
be incorrect if the test temporary directory was redirected (e.g.
via NODE_TEST_DIR) outside of `test/`.

Signed-off-by: Richard Lau <riclau@uk.ibm.com>

Backport-PR-URL: #32642
PR-URL: #32645
Refs: #32518
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Shelley Vohr <codebytere@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@BethGriggs
Copy link
Member

Landed in 2e3d511...017909b

@BethGriggs BethGriggs closed this Apr 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI. doc Issues and PRs related to the documentations. tools Issues and PRs related to the tools directory.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants