Pin PR CI tests against specific substrate; daily compat tests against latest master.

[jsdw]

Following #329, we could also pin the integration tests carried out in CI to a specific version of substrate, so that breaking changes in substrate don't start breaking out PRs here.

We still want to know about such breaking changes, and for this we can add a new CI file that does scheduled daily runs like:

name: Daily compatibility check against substrate master
on:
  schedule:    
    - cron: "0 9 * * *"

There would be a lot of code duplication if we copied all of the CI steps over (you can import jobs I think from other files), but it is probably sufficient just to run a daily cargo test against latest substrate.

I'm not yet sure how a cron job failure here would be reported to us; I assume as a red tick on master. We'd need to make sure it was fairly obvious and easy to receive notifications for.

[niklasad1]

I think it would be reasonable to create an issue by the workflow/cronjob if the job failed such as cargo test failed <some err> and assign owners the of repo (probably you and Andrew to the ticket)

[jsdw]

Yup, I agree; I love it!

[jsdw]

The CI is in place to do this now; we just need to pick a substrate binary to pin to. The difficulty is that there aren't readily available binaries to point at at the mo, so nightly and PR runs both use latest substrate.

[jsdw]

I'm going to close this for now; we have a nightly run against latest substrate to pick up compatibility breaks, and while it would be nice to pin PR runs to a "known" substrate so that we can choose when to upgrade, I think we can live with it how it is a bit and open an ew issue for that if we decide we want it.