Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recipe era5_recipe.yml needs some fixes #1889

Open
SarahAlidoost opened this issue Oct 27, 2020 · 1 comment
Open

Recipe era5_recipe.yml needs some fixes #1889

SarahAlidoost opened this issue Oct 27, 2020 · 1 comment

Comments

@SarahAlidoost
Copy link
Contributor

The recipe_era5.yml has several issues:

  1. The ERA5_freq and ERA5_name can be removed. However, I think ERA5_name gives some uesful information and I like to keep it. See this issue.
  2. The description should include the purpose of the recipe. For example, users working with hourly/monthly era5 data do not need to run this recipe prior to their own recipe. This recipe should be considered as an example of how to write their own recipe. The current hourly diagnostic may lead to memory and disk space issues.
  3. Monthly diagnostic including the variable zg fails with this error:
2020-10-26 08:40:49,272 UTC [3351] ERROR   Failed to run fix_metadata([<iris 'Cube' of geopotential / (m**2 s**-2) (time: 12; pressure_level: 37; latitude: 721; longitude: 1440)>, <iris 'Cube' of geopotential / (m**2 s**-2) (time: 12; latitude: 721; longitude: 1440)>, <iris 'Cube' of geopotential / (m**2 s**-2) (time: 12; latitude: 721; longitude: 1440)>, <iris 'Cube' of geopotential / (m**2 s**-2) (time: 12; pressure_level: 37; latitude: 721; longitude: 1440)>], {'project': 'native6', 'dataset': 'ERA5', 'short_name': 'zg', 'mip': 'Amon', 'frequency': 'mon', 'check_level': <CheckLevels.DEFAULT: 3>})
2020-10-26 08:41:11,228 UTC [3339] INFO    Maximum memory used (estimate): 5.2 GB
2020-10-26 08:41:11,229 UTC [3339] INFO    Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
2020-10-26 08:41:11,231 UTC [3339] ERROR   Program terminated abnormally, see stack trace below for more information
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/esmvalcore/_task.py", line 735, in _run_task
    output_files = task.run()
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/esmvalcore/_task.py", line 242, in run
    self.output_files = self._run(input_files)
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/esmvalcore/preprocessor/__init__.py", line 428, in _run
    product.apply(step, self.debug)
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/esmvalcore/preprocessor/__init__.py", line 296, in apply
    self.cubes = preprocess(self.cubes, step, **self.settings[step])
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/esmvalcore/preprocessor/__init__.py", line 238, in preprocess
    result.append(_run_preproc_function(function, items, settings))
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/esmvalcore/preprocessor/__init__.py", line 224, in _run_preproc_function
    return function(items, **kwargs)
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/esmvalcore/cmor/fix.py", line 107, in fix_metadata
    cube_list = fix.fix_metadata(cube_list)
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/esmvalcore/cmor/_fixes/native6/era5.py", line 325, in fix_metadata
    cube = self._fix_coordinates(cube)
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/esmvalcore/cmor/_fixes/native6/era5.py", line 277, in _fix_coordinates
    coord = cube.coord(axis=axis)
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/iris/cube.py", line 1497, in coord
    raise iris.exceptions.CoordinateNotFoundError(msg)
iris.exceptions.CoordinateNotFoundError: 'Expected to find exactly 1  coordinate, but found none.'
"""

4- Hourly diagnostic including the variable orog fails with this error:

2020-10-27 10:21:32,897 UTC [27684] INFO    These tasks will be executed: hourly/rename, hourly/orog
2020-10-27 10:21:32,899 UTC [27684] INFO    Running 2 tasks using 2 processes
2020-10-27 10:21:32,936 UTC [27706] INFO    Starting task hourly/orog in process [27706]
2020-10-27 10:21:33,033 UTC [27684] INFO    Progress: 1 tasks running, 1 tasks waiting for ancestors, 0/2 done
2020-10-27 10:21:33,466 UTC [27706] ERROR   Failed to run concatenate([<iris 'Cube' of surface_altitude / (m) (latitude: 721; longitude: 1440)>, <iris 'Cube' of surface_altitude / (m) (latitude: 721; longitude: 1440)>, <iris 'Cube' of surface_altitude / (m) (latitude: 721; longitude: 1440)>, <iris 'Cube' of surface_altitude / (m) (latitude: 721; longitude: 1440)>], {})
2020-10-27 10:21:33,715 UTC [27684] INFO    Maximum memory used (estimate): 0.0 GB
2020-10-27 10:21:33,716 UTC [27684] INFO    Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
2020-10-27 10:21:33,718 UTC [27684] ERROR   Program terminated abnormally, see stack trace below for more information
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/esmvalcore/preprocessor/_io.py", line 185, in concatenate
    cubes = sorted(cubes, key=lambda c: c.coord("time").cell(0).point)
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/esmvalcore/preprocessor/_io.py", line 185, in <lambda>
    cubes = sorted(cubes, key=lambda c: c.coord("time").cell(0).point)
  File "/home/fakhereh/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/iris/cube.py", line 1497, in coord
    raise iris.exceptions.CoordinateNotFoundError(msg)
iris.exceptions.CoordinateNotFoundError: 'Expected to find exactly 1 time coordinate, but found none.'
@SarahAlidoost
Copy link
Contributor Author

There is a bug in cmorizer script era5.py at this line. It needs a fix as:

        if info['diagnostic'] == 'daily':
            for mip in ['day', 'Eday', 'CFday']:
                if CMOR_TABLES['CMIP6'].get_variable(mip, info['short_name']):
                    basename = basename.replace('E1hr', mip)
                 else:
                    basename = basename.replace('E1hr', 'day')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant