Comprehensive CI tests on min/max supported versions of dependencies

[weiji14]

Modify CI test matrix to run on two streams - a minimum version stream and maximum version stream. Follow up of #914.

Specifically, the two streams are:

• Minimum stream:
  • Python=3.7
  • Pytorch=1.7.0
  • Torchvision=0.8.0
  • mmcv-full=1.3.3
  • mmdet=2.12.0
• Maximum stream:
  • Python=3.8
  • Pytorch=1.9.0
  • Torchvision=0.10.0
  • mmcv-full=1.3.12
  • mmdet=2.16.0

Feel free to suggest modifications and/or include more versions of other dependencies (e.g. those listed at https://github.com/airctic/icevision/blob/0.8.1/setup.cfg#L28-L39). I can also include a 'middle' version stream to test e.g. Pytorch 1.8.0, torchvision 0.9.0, etc if that would be better.

References:

• https://docs.github.com/en/actions/learn-github-actions/managing-complex-workflows
• https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#example-including-additional-values-into-combinations
• https://github.com/open-mmlab/mmcv/tree/v1.3.12#installation
• https://github.com/open-mmlab/mmdetection/blob/v2.16.0/docs/get_started.md#prerequisites

[lgvaz]

include more versions of other dependencies

That's a good question to ask, I think fastai is a good addition, since if often produces breaking changes with pytorch upgrades.

---

Do you know why the current tests are failing?

---

Is there a way for these stream to show up a separate items in the workflow UI?

[ai-fast-track]

@weiji14 @lgvaz I spent a lot time figuring out how to make all the dependencies play nicely with each other, that's how I updated the install_cola.sh file.

For example, if we don't have the right fastai version, the latter might install another version of both torch and torchvision: That might affect mmdet and mmcv versions.

Base on that experience, I think we need to pin the versions for fastai, pytorch-lightning, albumentations

Here is my last experiment in updating the install_cola.sh file (no pushed yet) for:

torch  1.9.0+cu102
torchvision  0.10.0+cu102
torchtext  0.10.0

We need to pin the following dependencies:

fastai  2.5.2
albumentations  0.5.2
pytorch-lightning  1.4.5

The mmcv and mmdet compatible versions will be:

mmcv  1.3.12
mmdet  2.16.0

MMLabs have a pypi package (openmim ) that automagically installs the right versions of both mmcv and mmdet without providing torch and cuda versions. That's how I'm using it:

pip install openmim -q
mim install mmcv-full
mim install mmdet

[ai-fast-track]

Do you know why the current tests are failing?

Is there a way for these stream to show up a separate items in the workflow UI?

Using mim seems to solve that problem based on the issue in MMDet repo here: open-mmlab/mmdetection#5367

[weiji14]

Using mim seems to solve that problem based on the issue in MMDet repo here: open-mmlab/mmdetection#5367

Ok, I've updated the workflow to use mmcv-full=1.3.13 which seems to include the fix according to open-mmlab/mmdetection#5367 (comment). Will see if the tests pass.

Is there a way for these stream to show up a separate items in the workflow UI?

They're showing up as two separate items no? tests / build (ubuntu-20.04, 3.7) and tests / build (ubuntu-20.04, 3.8).

Might wait for #919 to get merged since the updated setup.cfg file might affect things here...

[weiji14]

MMLabs have a pypi package (openmim ) that automagically installs the right versions of both mmcv and mmdet without providing torch and cuda versions. That's how I'm using it:

pip install openmim -q
mim install mmcv-full
mim install mmdet

Thanks for chipping in @ai-fast-track! So do you suggest doing something like this? And do I need to add a pip install fastai line here (before pip install -e .[all,dev])?

Suggested change

	pip install torch==${{ matrix.torch-version }} torchvision==${{ matrix.torchvision-version }} -f https://download.pytorch.org/whl/torch_stable.html
	pip install mmcv-full==${{ matrix.mmcv-full-version }} -f https://download.openmmlab.com/mmcv/dist/cpu/torch${{ matrix.torch-version }}/index.html
	pip install mmdet==${{ matrix.mmdet-version }}
	pip install -e ".[all,dev]"
	pip install yolov5-icevision --upgrade
	pip install torch==${{ matrix.torch-version }} torchvision==${{ matrix.torchvision-version }} -f https://download.pytorch.org/whl/torch_stable.html
	pip install openmim
	mim install mmcv-full==${{ matrix.mmcv-full-version }}
	mim install mmdet==${{ matrix.mmdet-version }}
	pip install -e ".[all,dev]"
	pip install yolov5-icevision --upgrade