Automatically update tools in Rust repo

[nrc]

It is a hassle and takes ages (because of the bors queue) to update tools in the Rust repo. I would like something more automated. I envisage a tool which:

• runs daily (or maybe weekly, whatever)
• only works on master (not beta channel, etc)
• checks toolstate.toml and the CI for the tool and only updates if both are passing tests
• creates its own branch where it:
  • updates the tools submodules
  • runs the complete test suite (but does not interrupt the bors queue to do so)
  • if we pass, merges at the next opportunity (i.e., we don't interrupt another Rust PR, but we don't wait to re-test, we should check toolstate.toml though)

This should mean tools on nightly are always somewhat up to date. We will not slow down the bors queue in any way. However, it does mean we are merging PRs which have not been fully tested. However, I hypothesise that because we only touch the tools submodules and we wouldn't merge if another PR touches the submodules or toolstate.toml, breakage would be extremely rare.

[oli-obk]

I have an idea for an intermediate step. If travis passes a PR that just modifies toolstate.toml, Cargo.lock and submodules, just merge the PR without running all of bors (after r+ of course).

[nrc]

Yeah, I would like to do that too! We've always had some push back about landing PRs without full bors testing, but just updating a submodule seems like it might be a good case. It is possible that updating toolstate.toml could cause problems - e.g., there is a PR which changes broken to testing for a tool and updates a submodule to fix the cause. This passes Travis. However, before landing another PR breaks the tool in a different way (and doesn't need to touch toolstate.toml because the tool is already broken). Then the tool PR would fail CI, but is landed without running the CI.

Another incremental approach might be only running the tool tests - e.g., when landing a rustfmt change (including toolstate.toml) we only run ./x.py test src/tools/whatever rather than ./x.py test (or dist or whatever).