Using Dependabot to manage doc build and CI versions

[jthielen]

As brought up on the bi-weekly community developers meeting, it sounds like Pandas v1.1.0 is breaking doc builds on RTD. One solution to the issues of frequent breakages in doc builds and CI due to upstream updates is having fixed version lists for all of these, which are then incrementally updated as new versions come out. @dopplershift has done a lot of great work in MetPy getting such a workflow set up with Dependabot (Unidata/MetPy#1410) among other CI updates, and this could be adapted for use here in xarray.

We've generally been quite happy with our updated CI configuration with Dependabot over the past couple weeks. The only major issue has been Unidata/MetPy#1424 / dependabot/dependabot-core#2198 (comment), which has required some contributors to have to delete and recreate their forks in order for Dependabot to not auto-submit PRs to the forked repos.

Any thoughts that you had here @dopplershift would be appreciated!

xref #4287, #4296

[dopplershift]

So on MetPy we moved to treating our CI system as an application and pinning every direct dependency in a requirements.txt (which can be used by conda as well). We then let dependabot handle the updates. This let's us manage the updates on a package-by-package basis, where we have a single PR that lets us see what the ramifications are with regards to tests, CI, even linting.

We've been running for a limited time, but so far it has done a good job of insulating general development (coming in on PRs) from changes in the environment, which now shouldn't change on CI from run to run (yeah, yeah 2nd-order dependencies, just pin problematic ones too). For instance, for the pandas 1.1.0 breakage, we just haven't merged the PR that moves the pin there, and that has kept our doc builds green on MetPy.

[shoyer]

This does seem like it would be really nice for us!

The main limitation compared to our current setup is that we currently ping dependencies using Conda's yml files, e.g., https://github.com/pydata/xarray/blob/master/ci/requirements/py36.yml

But it sounds like conda can use requirements.txt files, too, these days? In which case this is just a matter of migration.

[shoyer]

The other question is if dependabot can be configured to update a subset of specific requirements.txt files?

[dopplershift]

So to be clear, conda can use the --file argument to read those files and treat them as if they were dependency requirements passed on the command line. Not quite the same as an environment.yml, but I haven't had any problems so far. It's been really nice since dependabot doesn't understand the environment files.

So you can point dependabot to a directory, where for pypi it looks for files ending in .txt. You can make a subset by using a directory, or on MetPy we've remove the .txt from files we don't want dependabot to update. It feels like a bit of a hack, but it works.