ARM Travis tests running on Emulator == SLOW

[rengolin]

Just opening this ticket to keep a reminder, I'll look into getting a cloud instance to enable native tests on Arm hardware. I don't have access to ARMv6 or ARMv7 hardware but I can provide ARMv8 tests, which can emulate ARMv7 via AArch32 natively on the machines that support AArch32.

With the last meaningful ARMv6 machine being RaspberryPi 1 and with that all but long gone, I think we should be fine just testing v7 and v8.

What are the requirements for the client? Do we need to install a slave in the machine, like Jenkins, or is that done via SSH keys access?

[martin-frbg]

No idea actually, the current CI tests use whatever API is provided by github. I should be able to run ARMv7 (Asus Tinkerboard) and ARMv8 (Sony Xperia) tests locally when actually required so the CI addition from #1864 is mostly for convenience. One problem with integration could be that I do not have administrative control of the project and it is unclear if/when xianyi is available again.

[rengolin]

I see, that's not optimal.

I could use my cloud instance to keep a local loop and somehow exposing the results (webpage? webservice?) until we get travis to fire at it. Let me see what I can do on my side, and I'll ping you when I have something running, we can decide later.

Worst case scenario, we can try a CI on our personal repos, which we do have control. But would be good to have someone active in the community (like you) with admin access. :)

[martin-frbg]

Apparently there is a company (and service) named shippable that provides free CI on ARMV8a hardware http://blog.shippable.com/shippable-arm-packet-deliver-native-ci-cd-for-arm-architecture

[rengolin]

Interesting, would that work with Travis without need for admin access on the github project?

[martin-frbg]

As far as I can tell it is a Travis-like service that would need the same admin access to set up, I just came across it in another context and wanted to put that information somewhere.

[rengolin]

Got an instance running, @martin-frbg send me your public ssh keys so I can give you access to it. My email is name.suname@linaro.org.

[rengolin]

Given we don't have admin rights in the project, I'm thinking in using GitHub's API to track commits and pull requests, build a branch or a PR and then make the result available somehow.

Btw, it takes about 1/2 hour to build all four variants, what's the timeout on Travis? It's under 10 min for one variant, which we can make that the "CI loop" for pull requests, but do a full build + run benchmarks and BLAS-Testers on all TARGETs overnight.

[brada4]

It seems that it is 60min, also there is some monitor for 5min without output.
Probably some simple thing like buildbot can poll latest commit and trigger rebuild(s)

[rengolin]

Right, if/when we get Travis triggering build on that machine, 60 minutes should be plenty for build-tests.

I'll think about the overnight tests in separate, as they'll need lots of cores to build, but only one to test, so need to balance the resources in the right way.

[brada4]

You can set up PID file as a mutex and skip builds while one batch is churning. It just takes some days to tune for purpose.

[rengolin]

The problem is mixing pull request builds with overnight / CI builds, because PR needs to be done individually (request based), while CI can build multiple commits (time based). I'll probably have to create another, smaller, instance for the batch system, and run it once a day only.

[brada4]

Anything will be better than yesterday, you know.

[rengolin]

:)

[rengolin]

So, I got a server building and running BLAS-Tester:
https://openblas.ddns.net/

This should get us going, at least for the basic support. We can work the Travis triggers once we have admin rights on the project.