Where to find the correct frequency for ERA5 hourly variables?

[SarahAlidoost]

The default drs of native6 contains the parameter {frequency}. The drs is used for ERA5 data in recipe_era5.yml.
Where to find the correct frequency for hourly variables in the recipe?

[Peter9192]

Related to: ESMValGroup/ESMValCore#645 where this was first introduced.

[Peter9192]

I think the DRS was updated in #645, but the recipe was not updated then. So the recipe still contains the old DRS settings (using ERA5_freq and ERA5_name.

[SarahAlidoost]

Related to: ESMValGroup/ESMValCore#645 where this was first introduced.

@Peter9192 thank you for the link. Let me explain the issue better.

The default drs for native6 is: 'Tier{tier}/{dataset}/{latestversion}/{frequency}/{short_name}'. The {frequency} can be defined as one of the frequencies in cmip6_CV.

Also, I saw a comment about drs of era5 by @bouweandela here. However, esmvaltool does not accept this drs for the variables in the recipe_era5.yml; actually, all hourly variables should be in 1hrPt, whereas pr in 1hr and orog in fx. This does not make sense. Also, according to cmip6 tables, the variable rsdt has frequency 1hrCM. But esmvaltool expect it in 1hrPt. We can check this by running recipe_check_obs.yml.

[SarahAlidoost]

I think the DRS was updated in #645, but the recipe was not updated then. So the recipe still contains the old DRS settings (using ERA5_freq and ERA5_name.

Agree that recipe_era5.yml needs several fixes. I made another issue.

[Peter9192]

Okay, so if I understand it correctly: if you want to store the ERA5 data according to the default DRS, it is very unclear which values to use for the frequency tag. Right?

[bouweandela]

The ESMValCore determines the frequency of a variable by looking it up in the CMOR table, based on the project, mip table, and variable name provided in the recipe. However, if this does not give the correct frequency, it can be overridden by specifying the frequency in the recipe.

[SarahAlidoost]

Okay, so if I understand it correctly: if you want to store the ERA5 data according to the default DRS, it is very unclear which values to use for the frequency tag. Right?

Right.

[bouweandela]

all hourly variables should be in 1hrPt, whereas pr in 1hr

I think that would only be the case for variables that are sampled at specific points in time, not for accumulated ones like precipication and radiation, these are accumulated values over the past hour in ERA5 and we convert them to a flux by dividing by the time, so these are averaged over the hour so they should probably have frequency 1hr (without Pt).

[SarahAlidoost]

The ESMValCore determines the frequency of a variable by looking it up in the CMOR table, based on the project, mip table, and variable name provided in the recipe. However, if this does not give the correct frequency, it can be overridden by specifying the frequency in the recipe.

Thanks. So, we need to add the frequency to the dataset section in the recipe for each diagnostic, e.g. for hourly:

additional_datasets:
    - {dataset: ERA5, project: native6, type: reanaly, version: '1', tier: 3, frequency: 1hr, start_year: 1990, end_year: 1991}

Is it right?

[bouweandela]

Only if what is looked up from the CMOR table does not provide the correct frequency, but I think this is likely, because there are very few hourly variables in CMIP6, so then it will try to look it up in another mip table, but still try to keep the frequency from the mip and things may go wrong.

[SarahAlidoost]

all hourly variables should be in 1hrPt, whereas pr in 1hr

I think that would only be the case for variables that are sampled at specific points in time, not for accumulated ones like precipication and radiation, these are accumulated values over the past hour in ERA5 and we convert them to a flux by dividing by the time, so these are averaged over the hour so they should probably have frequency 1hr (without Pt).

I agree. However, esmvaltool does not accept them in 1hr by default. When I run recipe_check_obs.yml., it says that all variables should be in 1hrPt except the variable pr.

[bouweandela]

You might need to specify the frequency in that recipe too.

[bouweandela]

It might look more intuitive to specify the frequency in the variable section in the recipe, but with the dataset should work fine too.

[Peter9192]

all hourly variables should be in 1hrPt, whereas pr in 1hr

Just a note to avoid confusion: when you say 'should be', I think you mean 'ESMValTool seems to expect this, based on the error messages produced by recipe_check_obs', right? Whereas I would use 'should be' to state what I think ESMValTool should do, based on my own reasoning, regardless of what ESMValTool actually does...

I agree that specifying it in the recipe would be the best solution. But then we still need to figure out what 'should be' the right frequency, based on the meaning of 1hrPt etc.

[bouweandela]

I think you mean 'ESMValTool seems to expect this, based on the error messages produced by recipe_check_obs', right?

No

Whereas I would use 'should be' to state what I think ESMValTool should do, based on my own reasoning, regardless of what ESMValTool actually does...

Yes, me too, but you need to tell ESMValTool what the correct frequency is, because it cannot magically know this.

[Peter9192]

I think you mean 'ESMValTool seems to expect this, based on the error messages produced by recipe_check_obs', right?

No

I was quoting @SarahAlidoost... trying to make sure we're all on the same page ;-)