automated build tests

[stefangweichinger]

Does anyone plan to get automated build tests set up via travis on github?
I use that for a small project and assume this would be very helpful. Maybe I can help here.

[djmitche]

That would be really handy. In my day, we ran tests in Buildbot on Zmanda hardware across a variety of operating systems, but I assume that's gone now.

[chassell]

we really really need an installcheck run that works in-source-tree

[djmitche]

Just running make check would be a start (and that runs in the source tree, iirc?)

[chassell]

@djmitche Those are the closest we have to unit-tests, I suppose. They seem to be more (as I didn't know they were there!) to test the OS and its features and our use of them.

They run directly in the tree, true. I'd rather have installcheck run there if possible.

[djmitche]

It's sort of the nature of "installcheck" that it runs on an "install"ed version :)

Maybe a path to victory here is to run the make check tests in CI, and then work to migrate some of the more unit-y bits of the install checks into "regular" checks.

Getting make check to run would be of substantial value just in checking that Amanda builds, to start with. And the checks would do some basic syntax verification of all imported Perl. It's not much, but it's a lot better than nothing!

When I was at Zmanda, we used Buildbot for this, testing across a number of platforms. Setting that up would be a fair bit of work (but perhaps worthwhile to get a set of platforms representative of the customer base, rather than just the platforms available in the free CI offerings). Anyway, just building and running make check on Linux would again be a lot better than nothing.

[chassell]

I have made a wee script (bash) that uses the "overlay" kernel module (pretty widespread) and can make all installed disks everywhere appear to be writable but they are being "recorded" in an upper-layer transparent file system. Its sorta like having "docker" but no network isolation... though it does of course need root to set up. The idea, for now, is that changes we do want and safety we need should keep *all* the possible unit tests in place if possible. If we could devolve some of them down into unit-tests ... it would likely take some of these: * abstracting out the config so that it isn't permanently-and-only from /etc/amanda/... from a single-format-of-files etc..etc.. * using a PATH with a fake-installed-dir (i.e. /usr/lib64/amanda/ ...) and, similarly, a fake-logs-dir possible... without compromising security of a normal installation [i.e. don't sneak an environment var in and suddenly give restores to China]. * finally putting ALL THE NON-COMPILED PERL NEEDED for installcheck available in some fashion. We do need more unit-tests .. and device-specific unit tests would be wonderful as Zmanda is working with many new ones. On 9/16/19 8:57 PM, Dustin J. Mitchell wrote: It's sort of the nature of "installcheck" that it runs on an "install"ed version :) Maybe a path to victory here is to run the make check tests in CI, and then work to migrate some of the more unit-y bits of the install checks into "regular" checks. Getting make check to run would be of substantial value just in checking that Amanda builds, to start with. And the checks would do some basic syntax verification of all imported Perl. It's not much, but it's a lot better than nothing! When I was at Zmanda, we used Buildbot for this, testing across a number of platforms. Setting that up would be a fair bit of work (but perhaps worthwhile to get a set of platforms representative of the customer base, rather than just the platforms available in the free CI offerings). Anyway, just building and running make check on Linux would again be a lot better than nothing.