test: fix test-make-doc in CI #16301

Trott · 2017-10-18T19:19:42Z

First commit:

test: fix flaky test-make-doc

`test-make-doc` fails in CI on Raspberry Pi devices where `test-ci-js`
runs the test but does not build the docs. We do not want to build the
docs in these cases. Move the test to the `doctool` suite where it will
not run on CI.

Second commit:

build: run doctool test suite on CI

Run the doctool test suite on CI. This will only run on hosts that use
`make test-ci` but not `make test-ci-js`. The latter is for fanned hosts
that typically don't run builds themselves, so they will not have the
results of `make doc` available for `test-make-doc.js`.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
commit message follows commit guidelines

Affected core subsystem(s)

test

Trott · 2017-10-18T19:20:13Z

@nodejs/build

Trott · 2017-10-18T23:52:11Z

CI: https://ci.nodejs.org/job/node-test-pull-request/10837/

joyeecheung · 2017-10-19T01:57:12Z

Strange enough, looks like the test is still run on the Raspberry Pis

https://ci.nodejs.org/job/node-test-binary-arm/11054/RUN_SUBSET=4,label=pi1-raspbian-wheezy/console

not ok 253 doctool/test-make-doc
  ---
  duration_ms: 1.947
  severity: fail
  stack: |-
    fs.js:926
      return binding.readdir(pathModule.toNamespacedPath(path), options.encoding);
                     ^
    
    Error: ENOENT: no such file or directory, scandir '/home/iojs/build/workspace/node-test-binary-arm/out/doc/api'
        at Object.fs.readdirSync (fs.js:926:18)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-binary-arm/test/doctool/test-make-doc.js:15:17)
        at Module._compile (module.js:607:30)
        at Object.Module._extensions..js (module.js:618:10)
        at Module.load (module.js:526:32)
        at tryModuleLoad (module.js:489:12)
        at Function.Module._load (module.js:481:3)
        at Function.Module.runMain (module.js:648:10)
        at startup (bootstrap_node.js:191:16)
        at bootstrap_node.js:609:3

EDIT: and also other Raspberry Pi devices

`test-make-doc` fails in CI on Raspberry Pi devices where `test-ci-js` runs the test but does not build the docs. We do not want to build the docs in these cases. Move the test to the `doctool` suite and add that suite to `IGNORED_SUITES` in `test.py`. Specify `doctool` as a test suite to run with `make test` or `make test-ci` but not with the `make test-ci-js` job run on cross-compiled fanned CI tests like we do with Raspberry Pi devices in CI.

Trott · 2017-10-19T02:49:40Z

@joyeecheung Yeah, it's still running there because it's included in default suite for test.py. Looks like there's some cleanup to be done in our Makefile. I'm reworking this right now to work properly. Uncovering a lot of technical debt in the process. :-D

Trott · 2017-10-19T02:50:28Z

New approach:

test: fix flaky test-make-doc

`test-make-doc` fails in CI on Raspberry Pi devices where `test-ci-js`
runs the test but does not build the docs. We do not want to build the
docs in these cases. Move the test to the `doctool` suite and add that
suite to `IGNORED_SUITES` in `test.py`. Specify `doctool` as a test
suite to run with `make test` or `make test-ci` but not with the
`make test-ci-js` job run on cross-compiled fanned CI tests like we do
with Raspberry Pi devices in CI.

Trott · 2017-10-19T02:53:13Z

CI: https://ci.nodejs.org/job/node-test-pull-request/10843/

joyeecheung

CI is green % unrelated failures

Trott · 2017-10-19T16:08:25Z

Landed in c9d5be4.

`test-make-doc` fails in CI on Raspberry Pi devices where `test-ci-js` runs the test but does not build the docs. We do not want to build the docs in these cases. Move the test to the `doctool` suite and add that suite to `IGNORED_SUITES` in `test.py`. Specify `doctool` as a test suite to run with `make test` or `make test-ci` but not with the `make test-ci-js` job run on cross-compiled fanned CI tests like we do with Raspberry Pi devices in CI. PR-URL: nodejs#16301 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com>

`test-make-doc` fails in CI on Raspberry Pi devices where `test-ci-js` runs the test but does not build the docs. We do not want to build the docs in these cases. Move the test to the `doctool` suite and add that suite to `IGNORED_SUITES` in `test.py`. Specify `doctool` as a test suite to run with `make test` or `make test-ci` but not with the `make test-ci-js` job run on cross-compiled fanned CI tests like we do with Raspberry Pi devices in CI. PR-URL: #16301 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com>

`test-make-doc` fails in CI on Raspberry Pi devices where `test-ci-js` runs the test but does not build the docs. We do not want to build the docs in these cases. Move the test to the `doctool` suite and add that suite to `IGNORED_SUITES` in `test.py`. Specify `doctool` as a test suite to run with `make test` or `make test-ci` but not with the `make test-ci-js` job run on cross-compiled fanned CI tests like we do with Raspberry Pi devices in CI. PR-URL: nodejs/node#16301 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com>

MylesBorins · 2017-11-16T21:03:14Z

Should this be backported to v6.x-staging? If yes please follow the guide and raise a backport PR, if not let me know or add the dont-land-on label.

Trott · 2017-11-17T05:25:17Z

@MylesBorins If test-make-doc lands in v6.x, then this should. Otherwise, no.

`test-make-doc` fails in CI on Raspberry Pi devices where `test-ci-js` runs the test but does not build the docs. We do not want to build the docs in these cases. Move the test to the `doctool` suite and add that suite to `IGNORED_SUITES` in `test.py`. Specify `doctool` as a test suite to run with `make test` or `make test-ci` but not with the `make test-ci-js` job run on cross-compiled fanned CI tests like we do with Raspberry Pi devices in CI. PR-URL: nodejs/node#16301 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com>

Trott added the test Issues and PRs related to the tests. label Oct 18, 2017

nodejs-github-bot added the build Issues and PRs related to build files or the CI. label Oct 18, 2017

Trott added the wip Issues and PRs that are still a work in progress. label Oct 18, 2017

Trott force-pushed the fix-test-doc branch 2 times, most recently from a855967 to 20cbbf4 Compare October 18, 2017 20:44

Trott mentioned this pull request Oct 18, 2017

test: test make doc and verify toc #16208

Closed

3 tasks

Trott force-pushed the fix-test-doc branch from 20cbbf4 to 060c605 Compare October 19, 2017 02:50

refack approved these changes Oct 19, 2017

View reviewed changes

apapirovski approved these changes Oct 19, 2017

View reviewed changes

joyeecheung approved these changes Oct 19, 2017

View reviewed changes

joyeecheung mentioned this pull request Oct 19, 2017

build: use doc-only instead of doc #16309

Closed

2 tasks

AndreasMadsen mentioned this pull request Oct 19, 2017

async_hooks: skip sanity checks when disabled #15454

Closed

3 tasks

cjihrig approved these changes Oct 19, 2017

View reviewed changes

Trott closed this Oct 19, 2017

MylesBorins added the backport-requested-v6.x label Nov 16, 2017

Trott deleted the fix-test-doc branch January 13, 2022 22:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: fix test-make-doc in CI #16301

test: fix test-make-doc in CI #16301

Trott commented Oct 18, 2017 •

edited

Loading

Trott commented Oct 18, 2017

Trott commented Oct 18, 2017

joyeecheung commented Oct 19, 2017 •

edited

Loading

Trott commented Oct 19, 2017

Trott commented Oct 19, 2017

Trott commented Oct 19, 2017

joyeecheung left a comment

Trott commented Oct 19, 2017

MylesBorins commented Nov 16, 2017

Trott commented Nov 17, 2017

test: fix test-make-doc in CI #16301

test: fix test-make-doc in CI #16301

Conversation

Trott commented Oct 18, 2017 • edited Loading

Checklist

Affected core subsystem(s)

Trott commented Oct 18, 2017

Trott commented Oct 18, 2017

joyeecheung commented Oct 19, 2017 • edited Loading

Trott commented Oct 19, 2017

Trott commented Oct 19, 2017

Trott commented Oct 19, 2017

joyeecheung left a comment

Choose a reason for hiding this comment

Trott commented Oct 19, 2017

MylesBorins commented Nov 16, 2017

Trott commented Nov 17, 2017

Trott commented Oct 18, 2017 •

edited

Loading

joyeecheung commented Oct 19, 2017 •

edited

Loading