test: only skip slow tests on Raspberry Pi devices #42645

richardlau · 2022-04-07T21:05:12Z

Detect the Raspberry Pi devices in the Node.js CI and only skip the
slow tests on those instead of all armv7l devices.

For some more background we recently added an additional node-test-binary-armv7l job to node-test-commit-arm-fanned (nodejs/build#2914 (comment)) which runs the cross-compiled armv7l binaries in armv7l containers on the Ampere Altras donated to the project via the Works on Arm program at Equinix. These are fast enough to run the tests that are too slow to run on actual Pi hardware.

richardlau · 2022-04-07T21:07:41Z

test/common/index.js

+    // the contents of `/sys/firmware/devicetree/base/model` but that doesn't
+    // work inside a container. Match the chipset model number instead.
+    const cpuinfo = fs.readFileSync('/proc/cpuinfo', { encoding: 'utf8' });
+    return /^Hardware\s*:\s*(.*)$/im.exec(cpuinfo)?.[1] === 'BCM2835';


Note: BCM2835 matches what /proc/cpuinfo returns on the Pi 2's and 3's in our CI.
e.g.

Pi 3

$ docker exec node-ci-stretch cat /proc/cpuinfo processor : 0 model name : ARMv7 Processor rev 4 (v7l) BogoMIPS : 76.80 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x0 CPU part : 0xd03 CPU revision : 4 processor : 1 model name : ARMv7 Processor rev 4 (v7l) BogoMIPS : 76.80 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x0 CPU part : 0xd03 CPU revision : 4 processor : 2 model name : ARMv7 Processor rev 4 (v7l) BogoMIPS : 76.80 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x0 CPU part : 0xd03 CPU revision : 4 processor : 3 model name : ARMv7 Processor rev 4 (v7l) BogoMIPS : 76.80 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x0 CPU part : 0xd03 CPU revision : 4 Hardware : BCM2835 Revision : a22082 Serial : 000000001acb5e3f Model : Raspberry Pi 3 Model B Rev 1.2

Pi 2:

$ cat /proc/cpuinfo processor : 0 model name : ARMv7 Processor rev 5 (v7l) BogoMIPS : 38.40 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x0 CPU part : 0xc07 CPU revision : 5 processor : 1 model name : ARMv7 Processor rev 5 (v7l) BogoMIPS : 38.40 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x0 CPU part : 0xc07 CPU revision : 5 processor : 2 model name : ARMv7 Processor rev 5 (v7l) BogoMIPS : 38.40 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x0 CPU part : 0xc07 CPU revision : 5 processor : 3 model name : ARMv7 Processor rev 5 (v7l) BogoMIPS : 38.40 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x0 CPU part : 0xc07 CPU revision : 5 Hardware : BCM2835 Revision : a21041 Serial : 00000000e74d942b Model : Raspberry Pi 2 Model B Rev 1.1

I haven't matched against other Pi chipsets -- I don't know, for example, whether the Pi 4's are fast enough to run these slow tests (we don't have any in the Node.js CI).

Also os.cpus() doesn't return the Hardware field:

$ node -p "os.cpus()" [ { model: 'ARMv7 Processor rev 4 (v7l)', speed: 1200, times: { user: 5251030, nice: 0, sys: 634800, idle: 56307240, irq: 0 } }, { model: 'ARMv7 Processor rev 4 (v7l)', speed: 1200, times: { user: 5691040, nice: 0, sys: 1000630, idle: 56166190, irq: 0 } }, { model: 'ARMv7 Processor rev 4 (v7l)', speed: 1200, times: { user: 5880350, nice: 10, sys: 953020, idle: 55968650, irq: 0 } }, { model: 'ARMv7 Processor rev 4 (v7l)', speed: 1200, times: { user: 5741270, nice: 0, sys: 945350, idle: 56215820, irq: 0 } } ]

What is the problem with /sys? It's not available at all or that specific path doesn't exist or ?

@mscdex That specific path doesn't exist.

pi@test-requireio--williamkapke-debian10-arm64--pi3-3:~ $ docker exec node-ci-stretch ls -al /sys total 4 dr-xr-xr-x 12 root root 0 Apr 7 01:22 . drwxr-xr-x 20 root root 4096 Apr 7 01:21 .. drwxr-xr-x 2 root root 0 Apr 8 01:22 block drwxr-xr-x 26 root root 0 Apr 8 01:22 bus drwxr-xr-x 54 root root 0 Apr 8 01:22 class drwxr-xr-x 4 root root 0 Apr 8 01:22 dev drwxr-xr-x 11 root root 0 Apr 8 01:16 devices drwxrwxrwt 2 root root 40 Apr 7 01:22 firmware drwxr-xr-x 7 root root 0 Apr 7 01:22 fs drwxr-xr-x 12 root root 0 Apr 8 01:22 kernel drwxr-xr-x 124 root root 0 Apr 8 01:22 module drwxr-xr-x 2 root root 0 Apr 8 01:22 power pi@test-requireio--williamkapke-debian10-arm64--pi3-3:~ $ docker exec node-ci-stretch ls -al /sys/firmware total 0 drwxrwxrwt 2 root root 40 Apr 7 01:22 . dr-xr-xr-x 12 root root 0 Apr 7 01:22 .. pi@test-requireio--williamkapke-debian10-arm64--pi3-3:~ $

FWIW there should also be /proc/device-tree/model. Not sure if that's better or worse than parsing /proc/cpuinfo

/proc/device-tree appears to be a symlink inside the container pointing to a non-existent location:

pi@test-requireio--williamkapke-debian10-arm64--pi3-3:~ $ docker exec node-ci-stretch ls -al /proc/device-tree lrwxrwxrwx 1 root root 29 Apr 8 01:37 /proc/device-tree -> /sys/firmware/devicetree/base pi@test-requireio--williamkapke-debian10-arm64--pi3-3:~ $ docker exec node-ci-stretch ls -al /proc/device-tree/ ls: cannot access '/proc/device-tree/': No such file or directory pi@test-requireio--williamkapke-debian10-arm64--pi3-3:~ $

See also moby/moby#43419.

This won't stop it running on a pi4, but as you said that doesn't matter as we don't have any:

Hardware : BCM2711 Revision : d03114 Serial : 10000000302bfe33 Model : Raspberry Pi 4 Model B Rev 1.4

Similarly the ODROID distributions give their own custom output e.g.

Hardware : Hardkernel ODROID-XU4 Revision : 0100 Serial : 0000000000000000

(FYI I get timeouts in a couple of keepalive tests on that machine which is one of my own dev boxes, but no other timeouts based on the one run I've just done)

mhdawson

LGTM

nodejs-github-bot · 2022-04-08T00:59:31Z

CI: https://ci.nodejs.org/job/node-test-pull-request/43382/

nodejs-github-bot · 2022-04-08T02:42:08Z

CI: https://ci.nodejs.org/job/node-test-pull-request/43383/

nodejs-github-bot · 2022-04-08T03:21:01Z

CI: https://ci.nodejs.org/job/node-test-pull-request/43385/

targos · 2022-04-08T08:39:29Z

which runs the cross-compiled armv7l binaries in armv7l containers on the Ampere Altras donated to the project via the Works on Arm program at Equinix

I'm not sure I understand this, because Ampere Altra is based on ARMv8. Is there an emulation layer for armv7l ?

sxa · 2022-04-08T11:04:12Z

which runs the cross-compiled armv7l binaries in armv7l containers on the Ampere Altras donated to the project via the Works on Arm program at Equinix

I'm not sure I understand this, because Ampere Altra is based on ARMv8. Is there an emulation layer for armv7l ?

The Altras have the native 32-bit armv7l instruction set too so they execute the docker containers natively without emulation. Most armv8 systems do however it's not a requirement for the chips to include it (The previous ThunderX aarch64 systems we had were quite unusual in NOT having this support - which is why moving to the Altras was what allowed us to run these containers)

Surprisingly (at least to me) Apple's M1 CPUs also seem to include the armv7l instructions - I wouldn't have thought they'd have bothered ...

sxa

May be worth adjusting the messages etc. to clarify that it will affect the older generation pis only, but for the purposes of our CI I'm equally happy to leave this as-is.

Detect the Raspberry Pi devices in the Node.js CI and only skip the slow tests on those instead of all armv7l devices.

richardlau · 2022-04-11T17:07:31Z

Updated to pick up (and change) the new test from #42409.

nodejs-github-bot · 2022-04-11T17:18:52Z

CI: https://ci.nodejs.org/job/node-test-pull-request/43439/

nodejs-github-bot · 2022-04-12T01:07:16Z

CI: https://ci.nodejs.org/job/node-test-pull-request/43442/

Detect the Raspberry Pi devices in the Node.js CI and only skip the slow tests on those instead of all armv7l devices. PR-URL: #42645 Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Stewart X Addison <sxa@redhat.com> Reviewed-By: Darshan Sen <raisinten@gmail.com>

mhdawson · 2022-04-12T18:20:48Z

Landed in 73d53fe

Detect the Raspberry Pi devices in the Node.js CI and only skip the slow tests on those instead of all armv7l devices. PR-URL: nodejs#42645 Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Stewart X Addison <sxa@redhat.com> Reviewed-By: Darshan Sen <raisinten@gmail.com>

Detect the Raspberry Pi devices in the Node.js CI and only skip the slow tests on those instead of all armv7l devices. PR-URL: #42645 Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Stewart X Addison <sxa@redhat.com> Reviewed-By: Darshan Sen <raisinten@gmail.com>

Detect the Raspberry Pi devices in the Node.js CI and only skip the slow tests on those instead of all armv7l devices. PR-URL: nodejs/node#42645 Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Stewart X Addison <sxa@redhat.com> Reviewed-By: Darshan Sen <raisinten@gmail.com>

nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Apr 7, 2022

richardlau commented Apr 7, 2022

View reviewed changes

richardlau added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 7, 2022

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 7, 2022

This comment was marked as outdated.

Sign in to view

mhdawson approved these changes Apr 7, 2022

View reviewed changes

tniessen approved these changes Apr 7, 2022

View reviewed changes

This comment was marked as outdated.

Sign in to view

richardlau force-pushed the detectpi branch from 626728a to 574d013 Compare April 8, 2022 00:04

github-actions bot mentioned this pull request Apr 8, 2022

CI Reliability 2022-04-08 nodejs/reliability#245

Open

22 tasks

richardlau added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 8, 2022

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 8, 2022

richardlau requested a review from sxa April 8, 2022 11:13

sxa approved these changes Apr 8, 2022

View reviewed changes

RaisinTen approved these changes Apr 8, 2022

View reviewed changes

RaisinTen mentioned this pull request Apr 8, 2022

src,inspector: fix empty MaybeLocal crash #42409

Merged

richardlau added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Apr 8, 2022

This was referenced Apr 9, 2022

CI Reliability 2022-04-09 nodejs/reliability#246

Open

CI Reliability 2022-04-10 nodejs/reliability#247

Open

CI Reliability 2022-04-11 nodejs/reliability#248

Open

test: only skip slow tests on Raspberry Pi devices

9be705c

Detect the Raspberry Pi devices in the Node.js CI and only skip the slow tests on those instead of all armv7l devices.

richardlau force-pushed the detectpi branch from 574d013 to 9be705c Compare April 11, 2022 17:07

richardlau added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 11, 2022

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 11, 2022

github-actions bot mentioned this pull request Apr 12, 2022

CI Reliability 2022-04-12 nodejs/reliability#249

Open

29 tasks

mhdawson closed this Apr 12, 2022

richardlau deleted the detectpi branch April 12, 2022 18:21

github-actions bot mentioned this pull request Apr 13, 2022

CI Reliability 2022-04-13 nodejs/reliability#250

Open

25 tasks

github-actions bot mentioned this pull request Apr 14, 2022

CI Reliability 2022-04-14 nodejs/reliability#251

Open

24 tasks

BethGriggs mentioned this pull request Apr 14, 2022

2022-04-19, Version 18.0.0 (Current) #42262

Merged

This was referenced Apr 15, 2022

CI Reliability 2022-04-15 nodejs/reliability#252

Open

CI Reliability 2022-04-16 nodejs/reliability#253

Open

CI Reliability 2022-04-17 nodejs/reliability#254

Open

juanarbol mentioned this pull request May 31, 2022

2022-05-XX, Version 16.16.0 'Gallium' (LTS) #43268

Closed

targos mentioned this pull request Aug 3, 2022

v16.17.0 release proposal #44098

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: only skip slow tests on Raspberry Pi devices #42645

test: only skip slow tests on Raspberry Pi devices #42645

richardlau commented Apr 7, 2022

richardlau Apr 7, 2022 •

edited

richardlau Apr 7, 2022

mscdex Apr 8, 2022

richardlau Apr 8, 2022

mscdex Apr 8, 2022

richardlau Apr 8, 2022

richardlau Apr 8, 2022

sxa Apr 8, 2022

This comment was marked as outdated.

mhdawson left a comment

This comment was marked as outdated.

nodejs-github-bot commented Apr 8, 2022

nodejs-github-bot commented Apr 8, 2022

nodejs-github-bot commented Apr 8, 2022

targos commented Apr 8, 2022 •

edited

sxa commented Apr 8, 2022

sxa left a comment

richardlau commented Apr 11, 2022

nodejs-github-bot commented Apr 11, 2022

nodejs-github-bot commented Apr 12, 2022

mhdawson commented Apr 12, 2022

test: only skip slow tests on Raspberry Pi devices #42645

test: only skip slow tests on Raspberry Pi devices #42645

Conversation

richardlau commented Apr 7, 2022

richardlau Apr 7, 2022 • edited

Choose a reason for hiding this comment

richardlau Apr 7, 2022

Choose a reason for hiding this comment

mscdex Apr 8, 2022

Choose a reason for hiding this comment

richardlau Apr 8, 2022

Choose a reason for hiding this comment

mscdex Apr 8, 2022

Choose a reason for hiding this comment

richardlau Apr 8, 2022

Choose a reason for hiding this comment

richardlau Apr 8, 2022

Choose a reason for hiding this comment

sxa Apr 8, 2022

Choose a reason for hiding this comment

This comment was marked as outdated.

mhdawson left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

nodejs-github-bot commented Apr 8, 2022

nodejs-github-bot commented Apr 8, 2022

nodejs-github-bot commented Apr 8, 2022

targos commented Apr 8, 2022 • edited

sxa commented Apr 8, 2022

sxa left a comment

Choose a reason for hiding this comment

richardlau commented Apr 11, 2022

nodejs-github-bot commented Apr 11, 2022

nodejs-github-bot commented Apr 12, 2022

mhdawson commented Apr 12, 2022

richardlau Apr 7, 2022 •

edited

targos commented Apr 8, 2022 •

edited