Final test for v2.0.0 release

[mattiarighi]

A checklist of the ESMValTool recipes to be tested before the release

• recipe_era5.yml
• recipe_check_obs.yml
• recipe_concatenate_exps.yml
• recipe_correlation.yml
• recipe_extract_shape.yml
• recipe_julia.yml
• recipe_my_personal_diagnostic.yml
• recipe_ncl.yml
• recipe_preprocessor_derive_test.yml
• recipe_preprocessor_test.yml
• recipe_python_object_oriented.yml
• recipe_python.yml
• recipe_r.yml
• recipe_variable_groups.yml
• recipe_hype.yml
• recipe_lisflood.yml
• recipe_marrmot.yml
• recipe_pcrglobwb.yml
• recipe_wflow.yml
• recipe_albedolandcover.yml
• recipe_anav13jclim.yml
• recipe_arctic_ocean.yml
• recipe_autoassess_landsurface_permafrost.yml
• recipe_autoassess_landsurface_snow.yml
• recipe_autoassess_landsurface_soilmoisture.yml
• recipe_autoassess_landsurface_surfrad.yml
• recipe_autoassess_radiation_rms_Amon_all.yml
• recipe_autoassess_radiation_rms_Amon_obs.yml
• recipe_autoassess_radiation_rms_cfMon_all.yml
• recipe_autoassess_stratosphere.yml
• recipe_capacity_factor.yml
• recipe_carvalhais14nat.yml
• recipe_clouds_bias.yml
• recipe_clouds_ipcc.yml
• recipe_collins13ipcc.yml
• recipe_combined_indices.yml
• recipe_consecdrydays.yml
• recipe_cox18nature.yml
• recipe_cvdp.yml
• recipe_deangelis15nat.yml
• recipe_diurnal_temperature_index.yml
• recipe_ecs_constraints.yml
• recipe_ecs_scatter.yml
• recipe_ecs.yml
• recipe_ensclus.yml
• recipe_extreme_events.yml
• recipe_extreme_index.yml
• recipe_eyring06jgr.yml
• recipe_flato13ipcc.yml
• recipe_heatwaves_coldwaves.yml
• recipe_hyint_extreme_events.yml
• recipe_hyint.yml
• recipe_landcover.yml
• recipe_lauer13jclim.yml
• recipe_li17natcc.yml
• recipe_martin2018grl.yml
• recipe_miles_block.yml
• recipe_miles_eof.yml
• recipe_miles_regimes.yml
• recipe_modes_of_variability.yml
• recipe_multimodel_products.yml
• recipe_ocean_amoc.yml
• recipe_ocean_bgc.yml
• recipe_ocean_example.yml
• recipe_ocean_ice_extent.yml
• recipe_ocean_Landschuetzer2016.yml
• recipe_ocean_multimap.yml
• recipe_ocean_quadmap.yml
• recipe_ocean_scalar_fields.yml
• recipe_perfmetrics_CMIP5_4cds.yml
• recipe_perfmetrics_CMIP5.yml
• recipe_perfmetrics_land_CMIP5.yml
• recipe_quantilebias.yml
• recipe_rainfarm.yml
• recipe_runoff_et.yml
• recipe_russell18jgr.yml
• recipe_seaice_drift.yml
• recipe_seaice_feedback.yml
• recipe_seaice.yml
• recipe_shapeselect.yml
• recipe_smpi_4cds.yml
• recipe_smpi.yml
• recipe_snowalbedo.yml
• recipe_spei.yml
• recipe_tcr.yml
• recipe_thermodyn_diagtool.yml
• recipe_toymodel.yml
• recipe_validation_CMIP6.yml
• recipe_validation.yml
• recipe_wenzel14jgr.yml
• recipe_wenzel16jclim.yml
• recipe_wenzel16nat.yml
• recipe_williams09climdyn_CREM.yml (could only be tested with 1 dataset because of missing data)
• recipe_zmnam.yml

And here the same for the cmorizers:

• cmorize_obs_aphro_ma.py
• cmorize_obs_aura_tes.ncl
• cmorize_obs_berkeleyearth.py
• cmorize_obs_calipso_goccp.ncl
• cmorize_obs_cds_satellite_albedo.py
• cmorize_obs_cds_satellite_lai_fapar.py
• cmorize_obs_cds_satellite_soil_moisture.ncl
• cmorize_obs_cds_uerra.py
• cmorize_obs_cds_xch4.ncl
• cmorize_obs_cds_xco2.ncl
• cmorize_obs_ceres_ebaf.py
• cmorize_obs_ceres_syn1deg.ncl
• cmorize_obs_cowtanway.py
• cmorize_obs_cru.py
• cmorize_obs_ct2019.py
• cmorize_obs_duveiller2018.py
• cmorize_obs_e_obs.py
• cmorize_obs_eppley_vgpm_modis.py
• cmorize_obs_era_interim_land.py
• cmorize_obs_era_interim.py
• cmorize_obs_esacci_aerosol.ncl
• cmorize_obs_esacci_cloud.ncl
• cmorize_obs_esacci_fire.ncl
• cmorize_obs_esacci_landcover.ncl
• cmorize_obs_esacci_oc.py
• cmorize_obs_esacci_ozone.ncl
• cmorize_obs_esacci_soilmoisture.ncl
• cmorize_obs_esacci_sst.ncl
• cmorize_obs_ersl.ncl
• cmorize_obs_fluxcom.py
• cmorize_obs_gcp.py
• cmorize_obs_ghcn_cams.py
• cmorize_obs_ghcn.ncl
• cmorize_obs_gistemp.py
• cmorize_obs_gpcc.py
• cmorize_obs_hadcrut3.ncl
• cmorize_obs_hadcrut4.ncl
• cmorize_obs_hadisst.ncl
• cmorize_obs_haloe.ncl
• cmorize_obs_hwsd.py
• cmorize_obs_isccp_fh.ncl
• cmorize_obs_jma_transcom.py
• cmorize_obs_lai3g.py
• cmorize_obs_landflux_eval.py
• cmorize_obs_landschuetzer2016.py
• cmorize_obs_mac_lwp.ncl
• cmorize_obs_merra2.py
• cmorize_obs_mls_aura.py
• cmorize_obs_modis.ncl
• cmorize_obs_mte.py
• cmorize_obs_ncep.ncl
• cmorize_obs_ndp.py
• cmorize_obs_niwa_bs.ncl
• cmorize_obs_nsidc_0116_nh.py
• cmorize_obs_nsidc_0116_sh.py
• cmorize_obs_osi_450_nh.py
• cmorize_obs_osi_450_sh.py
• cmorize_obs_patmos_x.ncl
• cmorize_obs_persiann_cdr.py
• cmorize_obs_phc.py
• cmorize_obs_piomas.py
• cmorize_obs_regen.py
• cmorize_obs_uwisc.ncl
• cmorize_obs_woa.py

Should we also test the download scripts?

[mattiarighi]

@jvegasbsc how do you run the cmorizers with the new interface?

[valeriupredoi]

like this @mattiarighi (not sure about tcsh, this is for bash only though):

if [ $"day" = "Saturday" ]; then
    cmorizers="Mattia shouldn't work"
else
    cmorizers="Broken, see below"
fi

and the broken bit is because:

(esmvaltool_users) [valeriu@jasmin-sci1 ~]$ cmorize_obs -o WOA --config-file esmvaltool_var_test/config-user.yml 
Traceback (most recent call last):
  File "/home/users/valeriu/anaconda3R/envs/esmvaltool_users/bin/cmorize_obs", line 11, in <module>
    load_entry_point('ESMValTool', 'console_scripts', 'cmorize_obs')()
  File "/home/users/valeriu/esmvaltool_users/esmvaltool/cmorizers/obs/cmorize_obs.py", line 166, in main
    config_user = read_config_user_file(config_file, 'cmorize_obs')
TypeError: read_config_user_file() missing 1 required positional argument: 'options'

@jvegasbsc you need to make that arg optional for console scripts 🍺

[jvegreg]

I am on it, but in the meanwhile you can just past there an empty dict and it should work

[valeriupredoi]

cheers @jvegasbsc - why are we on GitHub on a Saturday? 🏖️

[mattiarighi]

We are getting there.
@bouweandela @valeriupredoi @jvegasbsc it would be good if you could test the recipes assigned to you, unfortunately I missed the data to do this myself.

@earnone @jhardenberg please fix the errors related to your recipes, @katjaweigel can also help with this.

@tomaslovato there are two of your cmorizers which are not working, it would be great if you could fix them.

I still have to test recipe_collins13ipcc.yml, but it requires lots of memory and the DKRZ job scheduler was down in the last few days. I will try again today.

[earnone]

@mattiarighi great. I understood #1752 by @katjaweigel solved the issues with recipe_hyint.yml (#1716), recipe_hyint_extreme_events.yml (#1711) and recipe_extreme_events.yml (#1718). So we are left with recipe_rainfarm.yml (#1715) @jhardenberg . OK?

[mattiarighi]

Correct.

[katjaweigel]

@earnone currently I'm just testing recipe_hyint.yml. Yesterday I had a version which run, but now something went wrong again when I tried to sort my changes into a new branch. Hope I have at least recipe_hyint.yml running again, soon.

[jvegreg]

Regarding my recipes, recipe_seaice_feedback already tested and recipe_seaice_drift is ongoing. I had to cmorize the NSIDC data, as we only have ERA5 and ERA-Interim in the Tier3 of the common data pool. This is intended, @valeriupredoi ? Should I put the data I cmorized there?

[jvegreg]

recipe_seaice_drift is ongoing

Finished ok

[valeriupredoi]

I will spend a good bit of next week making sure all my autoassess shtuff still works fine and is ready for the big launch 🚀 Dealing with some other annoyances, ahem, interesting phenomena in the meantime 😁

[jvegreg]

@bouweandela, I tried to run the hydrology recipes but I can not find where to get the shapefiles that they require. It is nor in the recipe nor in the documentation. Is there a reason why you are not telling where to get them?

[bouweandela]

@jvegasbsc In principle, the recipes are intended to work with any shapefile describing a river catchment. We recently discussed adding a link to the place where we downloaded them from to the recipe documentation, but this hasn't been done yet. Maybe @nielsdrost can say more about this?

Specifically, copies of the shapefiles mentioned in the recipes can be found here: https://github.com/eWaterCycle/recipes_auxiliary_datasets, but they are just examples.

[nielsdrost]

@jvegasbsc In principle, the recipes are intended to work with any shapefile describing a river catchment. We recently discussed adding a link to the place where we downloaded them from to the recipe documentation, but this hasn't been done yet. Maybe @nielsdrost can say more about this?

Specifically, copies of the shapefiles mentioned in the recipes can be found here: https://github.com/eWaterCycle/recipes_auxiliary_datasets, but they are just examples.

The hydrology recipes should in principle work with any shape. Usually these will come from HydroSHEDS (https://www.hydrosheds.org/), or some custom shape depending on the use-case. We though about including a few example shapes, but could not figure out how to best do this in ESMValTool. Committing them to the repo did not seem to be the best idea, but I also do not really regard them as a "dataset" worth publishing separately. Something for a future ESMValTool workshop discussion, I think.

In the mean time, shall we add a hint on how to get an example shapefile to the recipes?

[bouweandela]

In the mean time, shall we add a hint on how to get an example shapefile to the recipes?

Yes, I think it would be really helpful to put a link to HydroSHEDS in https://github.com/ESMValGroup/ESMValTool/blob/master/doc/sphinx/source/recipes/recipe_hydrology.rst. Maybe we could replace the names of shapefiles in the recipes so their names match a shapefile from that site?

[nielsdrost]

In the mean time, shall we add a hint on how to get an example shapefile to the recipes?

Yes, I think it would be really helpful to put a link to HydroSHEDS in https://github.com/ESMValGroup/ESMValTool/blob/master/doc/sphinx/source/recipes/recipe_hydrology.rst. Maybe we could replace the names of shapefiles in the recipes so their names match a shapefile from that site?

Created a separate issue (#1762)

[valeriupredoi]

tested the ones that had my name against them plus the two validation ones plus the preprocessor derive (note for the last one no OImon data for HadGEM2-A on Jasmin/BADC neither in output1 nor output2, maybe we should use a slightly less obscure dataset?) 🍺

[bouweandela]

preprocessor derive (note for the last one no OImon data for HadGEM2-A on Jasmin/BADC neither in output1 nor output2, maybe we should use a slightly less obscure dataset?)

I changed the mip of the last diagnostic in #1768, so now at least it runs, but just realized that it probably was the wrong way to fix it, because the description is Test sispeed, so probably the variable name was wrong too. A more likely test would be

  diag8:
    description: Test sispeed
    variables:
      sipeed:
        project: CMIP5
        mip: day
        exp: historical
        start_year: 1985
        end_year: 1986
        ensemble: r1i1p1
        derive: true
        force_derivation: false
        additional_datasets:
          - {dataset: MPI-ESM-MR}
    scripts: null

@jvegasbsc I think you originally added this. Is it OK to leave it as is? If not, can you fix it?

[jvegreg]

@jvegasbsc I think you originally added this. Is it OK to leave it as is? If not, can you fix it?

Fixed, waiting for the test to finish on Jasmin...

[mattiarighi]

We are almost there:

• @valeriupredoi any news on recipe_autoassess_radiation_rms_cfMon_all.yml?
• it looks like we can't fix recipe_ocean_example.yml and recipe_ocean_bgc.yml broken #1742 without a new ESMValCore release, is there a workaround?

[valeriupredoi]

yes - done, forgot to check the box 😁

[bouweandela]

it looks like we can't fix #1742 without a new ESMValCore release, is there a workaround?

We could try removing the multi model statistics from the recipes as a workaround, but it looks like @ledm is still working on these recipes in #1331 and #1621, so my impression is that this is not ready for the current release.

[mattiarighi]

That's bad, because it is in the Part II paper.
I would suggest to at least remove the multimodel stats if the recipe is running.