[WIP] beaker usually runs all tests on each provided os node

[tuxmea]

control-repo has another usecase:
run the test for each role onto a subset of os nodes

we disable beaker task and generate new ones based on available
acceptance tests
for all roles to test we use multitask instead of task

make papply puppet 5 aware

[alvagante]

Let me know when is ready to be merged

[tuxmea]

@alvagante
beaker has issues with hiera.yaml in environment as this file uses eyaml as default.
I am wondering whether we should:

• use yaml as default backend
• provide a private/public key pair within PSICK
• switch to another hiera.yaml for beaker based testing.
  Please let me know what you prefer.

[alvagante]

The problem is with hiera eyaml or with the missing keys?
I'd prefer to use by default eyaml, we may eventually ship a pair of sample keys, but it's a bad example to give, that's way I prefer the current approach, where keys are expected, but not present, and are not used since in hiera there's no currently encrypted entry.
This doesn't prevent users to add crypted entries with their own keys, and take care of distributing their eyaml keypais where they are needed.
In such cases a crypted value might be overridden (for development/test environments typicall) with a clear text one, so that eyaml keys are not needed and testing locally is possible.

So, it's not possible to make beaker work with eyaml without keys (and without encrypted entries), I'd use a hiera.yaml dedicated for Beaker, with the burden to keep it aligned to the main one.

[tuxmea]

I decided to go for the most simple approach: just generate a keypair. I know that in this case we can never test encrypted value lookup properly.
Tests still fail on postfix: /usr/sbin/postconf: fatal: parameter inet_interfaces: no local interface found for ::1

[alvagante]

Probably the configuration has to adapt to non ipv6 environments as probably the intances run by beaker. But that's another problem, I'd say, feel free to skip it (commenting or changing the default mail_class) or quickly fix the issue in the profile's template

[tuxmea]

Tests are now running directly on psick root folder.
I copied the classes tests from profile to root spec folder. These are now failing.
facterdb can not deal with facter 2.5:

• https://github.com/mcanevet/rspec-puppet-facts/pull/53/files
• Create facter 2.5 files to support the release of facter 2.5 voxpupuli/facterdb#44
  I have pinned facter to 2.4.0 until either one solution is available.

[alvagante]

Why moving profile spec tests in the main control repo dir?
I would place here only node/roles tests (http://rspec-puppet.com/documentation/hosts/)

[alvagante]

I'd keep the spec structure in the main dir, add in spec/ only nodes spec for all the roles we want to test (just checking for compilation on different OS of most or all the roles would be great).
And move back to the profile module the relevant spec tests.
We have already some scripting that iterates over the site dir to rspec test all the modules there. This can be another step in the ci pipelines

[tuxmea]

+1 for adding hosts unit and acceptance tests only to psick root and keeping the other unit tests inside the site modules.
I will remove the classes files shortly.