Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache subgrid files in preproc #41

Merged
merged 2 commits into from
Nov 26, 2024

Conversation

awnawab
Copy link
Contributor

@awnawab awnawab commented Nov 21, 2024

The output of the model is very sensitive to the bathymetry and obstruction coefficients. Previously, these were both computed by create_bathymetry, and the resulting bathymetry files were cached in nexus (https://get.ecmwf.int/#browse/browse:ecwam:data%2Fbathymetry%2Fv1%2Fbathymetry_O1280_nfre29_ETOPO1_e6c62ffa77b1815082065ffb241be879.gz). Preproc would then read this bathymetry file and generate the subgrid files. Repeatable results across different compilers and architectures were thus achievable as long as the same bathymetry file was used.

Now that grib output is enabled, create_bathymetry rather than preproc computes the grids, and outputs the subgrid files in grib. Therefore the subgrid files now also have to be cached in nexus in order to get repeatable results across different compilers. This PR makes the necessary changes to the scripts to achieve this.

Right now, I have only uploaded the bathymetry and subgrid files for the O48 grids to nexus. As this is a new caching approach they are in the v2 folder. Older versions of ecwam that don't use grib output will continue to use the bathymetries in the v1 folder. Once the PR is approved I will upload the bathymetries and subgrids for the O320, O640 and O1280 grids too.

This PR also adds single precision validation hashes for the O320 test case.

@awnawab
Copy link
Contributor Author

awnawab commented Nov 21, 2024

@jrbidlot and @jkousal32 could you also please take a look and tell me if you agree with this change to the runner scripts?

Copy link
Contributor

@jrbidlot jrbidlot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Ahmad, good point that now with grib format of the files, we can cached them and make sure only one flavour is ever used (dp).
The intention of moving to grib format was always to be more portable.

Copy link
Collaborator

@wdeconinck wdeconinck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @awnawab after our discussions offline everything is clear and good to go

@wdeconinck wdeconinck merged commit 0f70ac3 into ecmwf-ifs:develop Nov 26, 2024
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants