Cross compiling for ARM on travis?

[ralphtheninja]

Would it be possible to trigger prebuilts on travis that cross compile to arm?

See #300

cc @juliangruber

[juliangruber]

it appears to be definitely possible, see: travis-ci/travis-ci#3376

just involves some set up work. this we could then use to build for 32bit linux as well! cc @mafintosh

[crzidea]

Any update? I'm using leveldown on ARM servers, and it takes a long time everytime installing leveldown.

[ralphtheninja]

No updates sorry.

[mathiask88]

https://github.com/zeromq/zeromq.js/ is doing arm cross compilation with prebuild

[native-api]

According to travis-ci/travis-ci#2790, only cross-compiling to Android is available out of the box.

https://github.com/Pi4J/pi4j/blob/master/.travis.yml is an example showing it's possible to cross-compile for Pi with a downloaded "rpi-tools" package.

As per Running Travis CI tests on ARM architecture | tomaz.me, it's also possible to run tests with qemu.

Finally, https://bennuttall.com/piwheels-building-a-faster-python-package-repository-for-raspberry-pi-users/ and https://www.piwheels.hostedpi.com/ say that the piwheels project compile everything from PyPI (though, if the latter is correct, only for py3.4 and py3.5 -- looks strange, maybe the info is outdated). So all we need to do is provide an sdist that they can run pip wheel <pkg> on.

[ahdinosaur]

in case it's useful or interesting, i experimented with prebuild-cross, or using Docker to cross-compile prebuilds for Debian armhf and arm64. my use case is to use these modules on a Raspberry Pi running Debian on arm64.

[ahdinosaur]

hi @staltz, when you build for mobile devices, do you know what target is being compiled? (by "target", i mean a target triplet like here, or a gnu triple like here.) for example, in prebuild-cross, i was compiling for the gnu triplets arm-linux-gnueabihf and aarch64-linux-gnu, which probably correspond to arm-unknown-linux-gnueabihf, armv7-unknown-linux-gnueabihf and aarch64-unknown-linux-gnu. i'm curious if my approach of using Debian cross-compilers to compile for mobile is even possible, or if we might need something like dockcross to compile for more specific targets.

[ahdinosaur]

another confusing thing i discovered is that Node's process.arch has no distinction between the various flavors within a given ARM architecture, it's only either arm or arm64. so even if we publish ARM prebuilds, there's actually more ARM flavors than just arm and arm64, and we need a way to distinguish which prebuild corresponds to the current system.

[ralphtheninja]

another confusing thing i discovered is that Node's process.arch has no distinction between the various flavors within a given ARM architecture, it's only either arm or arm64. so even if we publish ARM prebuilds, there's actually more ARM flavors than just arm and arm64, and we need a way to distinguish which prebuild corresponds to the current system.

There's a thread discussing similar things for prebuild/prebuild-install prebuild/prebuild#174. It's almost the same use case, where node-gyp-build plays the same role as prebuild-install and where we aren't downloading anything, just checking local files.

This prebuild/prebuild#174 (comment) sounds especially interesting.

[ahdinosaur]

@ralphtheninja i'm not sure the arm version is enough. i have a feeling that if we want to support arm prebuilds for both a raspberry pi and an android phone, we need to distinguish between arm for linux and arm for android.

[ralphtheninja]

@ahdinosaur I agree. So how can we do this?

I guess it's easy to know this when you're building the binary, since you know the exact system by default. Which means it's also easy to encode this into e.g. the file name of the resulting binary.

The most immediate problem we have is when resolving the binary with node-gyp-build which now only can use process.arch to distinguish between arm and arm64 which you already described (I'm mainly rubber ducking while writing this, sorry for repeating).

Is there any other way that we can get this information from the system? Since it seems just using node isn't adequate enough.

[vweevers]

For distinguishing arm flavors, zeromq wraps prebuild-install with some logic we could borrow:

https://github.com/zeromq/zeromq.js/blob/09b22070d5b1929a2191bb91d5c4621bf36fe9bf/scripts/prebuild-install.js#L12-L18

It's missing differentiation of android vs linux. Could it be combined with process.platform === 'android'?

[ralphtheninja]

It's missing differentiation of android vs linux. Could it be combined with process.platform === 'android'?

Assuming we can do this. And checking process.arch for arm and arm64 then the folder layout could look something like:

$ tree
.
├── android-arm6
│   ├── electron-napi.node
│   └── node-napi.node
├── android-arm7
│   ├── electron-napi.node
│   └── node-napi.node
├── android-arm8
│   ├── electron-napi.node
│   └── node-napi.node
├── darwin-x64
│   ├── electron-napi.node
│   └── node-napi.node
├── linux-arm6
│   ├── electron-napi.node
│   └── node-napi.node
├── linux-arm7
│   ├── electron-napi.node
│   └── node-napi.node
├── linux-arm8
│   ├── electron-napi.node
│   └── node-napi.node
├── linux-x64
│   ├── electron-napi.node
│   └── node-napi.node
└── win32-x64
    ├── electron-napi.node
    └── node-napi.node

Where *-arm8 would correspond to arm64 and all others would be checking process.config.variables.arm_version.

This would mean a new major version of prebuildify and node-gyp-build which I'm guessing handles arch the same no matter if it's x64 or arm.

[ahdinosaur]

okay, i did the thing and re-wrote prebuildify-cross using dockcross: #572 😺

[vweevers]

Made build-arch to wrap os.arch() with the logic described above. We can use this in node-gyp-build.

@ralphtheninja I used "armv" as prefix rather than "arm", for compat with prebuild-install.

[ralphtheninja]

armv7 done.