A list of models I can't get to work in ESMValTool with area_statistics preprocessor (mean)

[ledm]

I don't know if this is useful for anyone else, but I generated a recipe that looks at every tos (Temperature at the ocean surface) dataset in the historical and Tier1 SSP datasets. Pretty much every model in CMIP6 has provided this dataset. Elsewhere in the the recipe, I take a spatial average using area_statistics preprocessor, operator: mean, which of course, requires the areacello Ofx dataset.

The recipe is:

    area_statistics:
      fx_variables:
      - {activity: CMIP,  grid: gn, mip: Ofx, short_name: areacello, exp: '*', ensemble: '*'}
      operator: mean

(which requires ESMValGroup/ESMValCore#1082).

I've been working through these and making minor fixes where necessary. However, some models simply need to be excluded at the moment. The following models can not be used for the reason supplied:

skip_models  = [      # because
    'AWI-CM-1-1-MR',  # triangular grid
    'AWI-ESM-1-1-LR', # triangular grid
    'BCC-CSM2-MR',    # No fx files.
    'CAMS-CSM1-0',    # No fx files.
    'CMCC-CM2-HR4',   # No fx files.
    'GISS-E2-1-G-CC', # No fx files.
    'GISS-E2-1-H',    # No fx files.
    'GFDL-CM4',       # High resolution  (1000x1000 grid). - also problem with lat/lon in areacello.
    'GFDL-ESM4',      #  problem with lat/lon in areacello.
    'IITM-ESM',       # No longitude bounds.
    'FGOALS',         # No longitude bounds in irregular grid.
    'NESM3',          # Branch time in parent is incorrect.
    'MIROC-ES2L',     # SSP data contains loads of other variables.
    'NorESM2-LM',     # Depth provided in weird units (density)
    'NorESM2-MM',     # Depth provided in weird units (density)
    'r13i1p1f2',      # UKESM r13 also doesn't work as it's got the wrong grid.
    ]

So, if you want to remove them from your recipe, run the following command in python, then copy paste the output to the command line:

fn2 = 'recipe.yml'
for m in skip_models:
    print("sed -e '/"+m+"/ s/^#*/#/' -i ", fn2)

Which will produce something that looks like:

sed -e '/MIROC-ES2L/ s/^#*/#/' -i  recipe.yml

This sed command prepends a hash # to every line where it finds the model name, removing the model from the list of datasets. Note that this only works if you use the one line per model in your datasets list.

[zklaus]

This really is a slurry of different problems and I am not sure how closely they are related. What I can see now:

• Irregular grids (AWI): The inability to deal with this is something that will improve in the tool in the future. One building block will be regridding, but it would also be great to improve to the point where we can natively aggregate over unstructured grids. That may require improvements in ESMValTool/Core and also Iris.
• Simply missing required FX files should be reported as (individual!) data problems.
• High resolution in itself should not be an issue. If there is a problem, it would be helpful to discuss this separately.
• Missing or wrong coordinates should be reported as data problems.
• Erroneous branch time should be reported as a data problem, though I am not clear on how this impacts this analysis.
• Likewise, it is not clear to me why the depth units of NorESM2 are a problem here. Because of this, it is not clear how to solve it. If this is a genuine problem, we should understand that better, perhaps first in a discussion.
• A wrong grid should of course be reported as a data problem.

Finally, there is the MIROC-ES2L where you write "SSP data contains loads of other variables". How is that a problem?

[ledm]

Yep, each of these datasets their own problems. If this is the wrong way to summarise a set of problems with CMIP6 datasets, then please close the issue. However, these are genuine problems that prevent these models from being included in ESMValTool multi model analyses of the ocean.

• Triangular grids can not currently be opened in Iris, so they need to be pre-pre-processed outside of ESMValTool.
• High resolution isn't a problem in itself, but it does typically require using a slower high memory on jasmin. The problem with GFDL-CM4 isn't the resolution though, it's the areacello lat/lon coordinates.
• The problem with the wrong branch time means that we can't link the pi-control to the historical- not important for all analyses, but it effectively prevents de-drifting.
• The NorESM2 reporting depth as pressure is a massive problem though. It's not a straight forward fix to convert pressure to depth. In addition, the depth-pressure relationship changes with time, so this means that the model land mask changes with time, which is a new and strange behaviour which prevents de-drifting. In addition
• The MIROC-ES2L SSP files contain multiple data values, which causes iris to fail to load the netcdf as a cube.

[zklaus]

Yep, each of these datasets their own problems. If this is the wrong way to summarise a set of problems with CMIP6 datasets, then please close the issue.

I think as a discussion, it's perfect. As an issue it's a bit difficult because the obvious way to address it (addressing every last bit that's mentioned) would take a long time and make it almost useless for tracking progress.

However, these are genuine problems that prevent these models from being included in ESMValTool multi-model analyses of the ocean.

Sure.

Triangular grids can not currently be opened in Iris, so they need to be pre-pre-processed outside of ESMValTool.

Not true.

python -c "import iris; c = iris.load_cube('/badc/cmip6/data/CMIP6/CMIP/AWI/AWI-CM-1-1-MR/historical/r1i1p1f1/Omon/tos/gn/v20181218/tos_Omon_AWI-CM-1-1-MR_historical_r1i1p1f1_gn_185001-185012.nc'); print(c)"

works perfectly well.

High resolution isn't a problem in itself, but it does typically require using a slower high memory [machine] on jasmin. The problem with GFDL-CM4 isn't the resolution though, it's the areacello lat/lon coordinates.

So a simple dataset problem. Please file as such.

The problem with the wrong branch time means that we can't link the pi-control to the historical- not important for all analyses, but it effectively prevents de-drifting.

Sure. So this really has nothing to do with FX files or areacello. Please file the appropriate dataset problem.

The NorESM2 reporting depth as pressure is a massive problem though. It's not a straightforward fix to convert pressure to depth.

I acknowledge that this presents challenges. But for a surface variable such as tos this should be immaterial. Why is that not the case?
In any case, it is not necessary to calculate the depth yourself, as it is published as (Omon, zhalfo) and (Omon, zfullo), so it doesn't matter how straightforward the calculation is. Nevertheless, there may be something to talk about here in terms of vertical regridding.

In addition, the depth-pressure relationship changes with time, so this means that the model land mask changes with time, which is a new and strange behavior that prevents de-drifting.

I could not find corroboration for this claim. Certainly, the land-sea mask is published as (Ofx, sftof) with no time dependence. Could you provide some more information on how the mask depends on time?

The MIROC-ES2L SSP files contain multiple data values, which causes iris to fail to load the netcdf as a cube.

I randomly tried a couple of MIROC-ES2L SSP files from tos and areacello; they didn't exhibit this problem. Could you please point me to a problematic file for further investigation?