test_open_mfdataset_list_attr is ran even when dask is not available

[ArchangeGabriel]

I’m currently packaging xarray (0.15.0) for ArchLinux, and I’m running the test suite with all our currently available packages (in addition to python-numpy, python-pandas and python-pytest of course):

python-netcdf4
python-scipy
python-cftime
python-bottleneck
python-matplotlib
python-seaborn
python-pint

I was greatly impressed at the automatic selection of tests depending on what is available on the system. :) I’m only seeing 10 tests failures (10 failed, 8057 passed, 1384 skipped, 518 xfailed, 277 xpassed, 5268 warnings), and amongst them one is happening because a test requiring dask is run even without it being installed: test_open_mfdataset_list_attr.

Corresponding test output:

________________________ test_open_mfdataset_list_attr _________________________

    @requires_netCDF4
    def test_open_mfdataset_list_attr():
        """
        Case when an attribute of type list differs across the multiple files
        """
        from netCDF4 import Dataset
    
        with create_tmp_files(2) as nfiles:
            for i in range(2):
                f = Dataset(nfiles[i], "w") 
                f.createDimension("x", 3)
                vlvar = f.createVariable("test_var", np.int32, ("x"))
                # here create an attribute as a list 
                vlvar.test_attr = [f"string a {i}", f"string b {i}"]
                vlvar[:] = np.arange(3)
                f.close()
            ds1 = open_dataset(nfiles[0])
            ds2 = open_dataset(nfiles[1])
            original = xr.concat([ds1, ds2], dim="x")
>           with xr.open_mfdataset(
                [nfiles[0], nfiles[1]], combine="nested", concat_dim="x"
            ) as actual:                                                                                                                                                                                                                                                       

xarray/tests/test_backends.py:2561: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _  
xarray/backends/api.py:908: in open_mfdataset
    datasets = [open_(p, **open_kwargs) for p in paths]
xarray/backends/api.py:908: in <listcomp>
    datasets = [open_(p, **open_kwargs) for p in paths]
xarray/backends/api.py:538: in open_dataset
    ds = maybe_decode_store(store)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _  

store = <xarray.backends.netCDF4_.NetCDF4DataStore object at 0x7f392aebc640>
lock = False

    def maybe_decode_store(store, lock=False):
        ds = conventions.decode_cf(
            store,
            mask_and_scale=mask_and_scale,
            decode_times=decode_times,
            concat_characters=concat_characters,
            decode_coords=decode_coords,
            drop_variables=drop_variables,
            use_cftime=use_cftime,
        )    
    
        _protect_dataset_variables_inplace(ds, cache)
    
        if chunks is not None:
>           from dask.base import tokenize
E           ModuleNotFoundError: No module named 'dask'

xarray/backends/api.py:459: ModuleNotFoundError

I think this test should thus not be selected if dask is not installed. ;)

[max-sixty]

Thanks for finding these @ArchangeGabriel

That one looks like a reasonably easy fix; we just need to surround that test with the standard requires dask decorator...

[mathause]

The test is here:

xarray/xarray/tests/test_backends.py

Line 2543 in 52ee5df

def test_open_mfdataset_list_attr():

there is probably no test environment that has netCDF4 but not dask

[ArchangeGabriel]

You mean in your test matrix? Then yes. Anyway, I’ve proposed a simple fix in #3780, not sure if you want to go a different route (like making the test work without dask, not sure if that makes any sense).

[mathause]

You mean in your test matrix?

Yes, I was wondering why this was not caught anywhere & I think #3780 is fine.

[ArchangeGabriel]

Well, you can’t test every combination of dependencies, especially with the large number of optional ones you have. ;)