Support CMIP3 #269
Merged
Support CMIP3 #269
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jvegreg
added
cmor
|
I have updated the branch in ESMValTool to keep the recipe and ported all the changes here. I can not test with data because I don't have access to CMIP3 in Jasmin. I already requested it, but I don't know when I will be able to use it @LisaBock, can you give it a try? |
|
One remark: mip selection in CMIP3 does not have information about the frequency. It must be added manually |
|
I tested it, but it stopped here: I waited now about 30 minutes but nothing happens anymore. |
|
Thanks Lisa. I now have access to CMIP3 data in JASMIN. I will take a look and let you know |
|
Tested in Jasmin with this recipe for datasets:
- &cmip3 {project: CMIP3, exp: 20c3m, dataset: BCM2, ensemble: r1, start_year: 1979, end_year: 1990}
- {<<: *cmip3, dataset: CGCM3-1-T47}
- {<<: *cmip3, dataset: CGCM3-1-T63}
- {<<: *cmip3, dataset: BCC-CM1}
- {<<: *cmip3, dataset: CM3, institute: INM}
- {<<: *cmip3, dataset: CM3, institute: CNRM}
- {<<: *cmip3, dataset: MK3}
- {<<: *cmip3, dataset: MK3-5}
- {<<: *cmip3, dataset: CM2}
- {<<: *cmip3, dataset: CM2-1}
- {<<: *cmip3, dataset: ECHAM4}
- {<<: *cmip3, dataset: CM4}
- {<<: *cmip3, dataset: FGOALS-G1-0}
- {<<: *cmip3, dataset: ECHO-G}
- {<<: *cmip3, dataset: ECHAM5}
- {<<: *cmip3, dataset: GISS-AOM}
- {<<: *cmip3, dataset: GISS-EH}
- {<<: *cmip3, dataset: GISS-ER}
- {<<: *cmip3, dataset: CCSM3}
- {<<: *cmip3, dataset: PCM}
- {<<: *cmip3, dataset: MIROC3-2-HI}
- {<<: *cmip3, dataset: MIROC3-2-MED}
- {<<: *cmip3, dataset: HADCM3}
- {<<: *cmip3, dataset: HADGEM1}
preprocessors:
clim:
climate_statistics: r period: 'season'
Cdiagnostics:
sit_anom:
description: Computes the negative seaice feedback 2 variables:
zg:
mip: A1
frequency: mon
modeling_realm: atmos
preprocessor: clim
scripts: null@LisaBock, can you test the same on DKRZ and let me know what happens? |
|
The version of our CMIP3 data seems to be a different one. The names of our models are I think it is not that big problem but I can't use the institute table you defined in |
|
Should this step take longer? |
This will give us trouble when trying to run the same recipes at diferent data centers |
No, I was able to run the recipe quite fast. Never notice any slowdown on any file. If you try to run again, the bahaviour is the same? |
Yes, I tried it now severel times and it started with different models but it always stopped there. |
|
Ok. Can you put the file in a ftp or something, so I can try to debug it locally? |
|
Here you could find two of our CMIP3 datasets for miroc3_2_hires: tas_A1.nc cccma_cgcm3_1: tas_a1_20c3m_1_cgcm3.1_t47_1850_2000.nc |
|
Thanks! |
|
I can not reproduce the error locally and I don't have access to DKRZ. Maybe you can try to get help from @mattiarighi or @schlunma. If they also have the same problem maybe we can collect enough info to learn what it is really happening. |
bouweandela
reviewed
Oct 15, 2019
bouweandela
reviewed
Oct 15, 2019
7 tasks
bouweandela
requested changes
Dec 20, 2019
There was a problem hiding this comment.
Could you add a test recipe similar to this one?
ESMValCore/tests/integration/test_recipe.py
Line 550 in 74d251d
|
Done! |
bouweandela
approved these changes
Dec 20, 2019
|
I tried reading but |
bouweandela
reviewed
Jan 3, 2020
bouweandela
reviewed
Jan 3, 2020
Co-Authored-By: Bouwe Andela <bouweandela@users.noreply.github.com>
|
Tested again, with This is strange, since for these models there is nothing to concatenate: |
|
From @LisaBock :
Are you using one or more processes? |
|
Sorry, I missed that comment. Tried again with 1 processor and |
|
This looks like you have not set the drs properly in your config-user And its finding every file for each dataset: |
|
I'm collecting stupid mistakes today... 😞 I'm going to open an issue for the 1-processor problem, we should not forget it. |
|
also |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request aims to add support for CMIP3, which is required for comparison works between different epochs of the CMIP project.
Working with @LisaBock
Closes #27