Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add csawmg debug test, add to rt_gnu.conf #552

Closed
climbfuji opened this issue Apr 30, 2021 · 14 comments · Fixed by #2180
Closed

Add csawmg debug test, add to rt_gnu.conf #552

climbfuji opened this issue Apr 30, 2021 · 14 comments · Fixed by #2180
Labels
enhancement New feature or request

Comments

@climbfuji
Copy link
Collaborator

Description

csawmg-based jobs never ran with GNU, crashed with segmentation faults. Now that we have assumed-size arrays in CCPP (#527), we should add a DEBUG test to rt.conf and one PROD+DEBUG test to rt_gnu.conf.

When testing #527, I ran fv3_csawmg with Intel in DEBUB mode on Jet, and it worked.

@DeniseWorthen
Copy link
Collaborator

@climbfuji It looks like intel has both prod and debug tests now, but this still hasn't been done for GNU. I tried adding both a prod and debug to the rt_gnu.conf. Both segfault. The control_csamwg_debug test produces this starting traceback:

See /scratch1/NCEPDEV/stmp2/Denise.Worthen/FV3_RT/rt_304214/control_csawmg_debug

143:
143: Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.
143:
143: Backtrace for this error:
 38: #0  0x2b08ec45a3ff in ???
 55: #0  0x2b8f3693a3ff in ???
 58: #0  0x2acbd4b4d3ff in ???
 61: #0  0x2b32510463ff in ???
113: #0  0x2ab6ea0743ff in ???
115: #0  0x2ac3a79e63ff in ???
116: #0  0x2afc706d73ff in ???
121: #0  0x2ac0adea63ff in ???
 76: #0  0x2b7cbd73d3ff in ???
127: #0  0x2ab0590da3ff in ???
128: #0  0x2b6a8a5cb3ff in ???
 77: #0  0x2b1cc202a3ff in ???
130: #0  0x2b5a5eb483ff in ???
 93: #0  0x2abd2cee83ff in ???
133: #0  0x2b9b1dc033ff in ???
112: #0  0x2af7d46803ff in ???
143: #0  0x2b4ab79393ff in ???
 93: #1  0x27c6f36 in __tp_core_mod_MOD_pert_ppm
 93:    at /scratch2/NCEPDEV/climate/Denise.Worthen/WORK/ufs-weather-model-gnu/FV3/atmos_cubed_sphere/model/tp_core.F90:1199
 93: #2  0x27f1c60 in xppm
 93:    at /scratch2/NCEPDEV/climate/Denise.Worthen/WORK/ufs-weather-model-gnu/FV3/atmos_cubed_sphere/model/tp_core.F90:632
 93: #3  0x27f976a in __tp_core_mod_MOD_fv_tp_2d
 93:    at /scratch2/NCEPDEV/climate/Denise.Worthen/WORK/ufs-weather-model-gnu/FV3/atmos_cubed_sphere/model/tp_core.F90:181

@climbfuji
Copy link
Collaborator Author

climbfuji commented Aug 5, 2021

Update 08/05/2021: PR #724 fixes the aerosol options that led to recent problems with the csawmg based regression tests. The two tests control_csawmg and control_csawmgt now run in DEBUG and PROD mode with Intel on all platforms, but crash with GNU. This needs to be fixed, we shouldn't have code with known problems in the repository and not take any action.

See also the discussion in #724 for what was tried so far.

pjpegion pushed a commit to NOAA-PSL/ufs-weather-model that referenced this issue Apr 4, 2023
… temperatures (ufs-community#552)

* typo caused potential temperature calculation to use exponentiation instead of multiplication
epic-cicd-jenkins pushed a commit that referenced this issue Apr 17, 2023
## DESCRIPTION OF CHANGES: 
1. Add if condition in tests/run_experiments.sh so to get informative error message when MET and MET paths are not available on a machine.
2. Remove MET/MET+ paths in tests/baseline_configs/config.verification.sh

## TESTS CONDUCTED: 
A test run was conducted on Hera and has been finished successfully.  Another test was run on Jet, and it failed in the expected way.

## DEPENDENCIES:
To have MET verification run successfully, the observational data (e.g., CCPA, MRMS, NDAS) must be available.

## DOCUMENTATION:
N/A

## ISSUE (optional): 
This is a follow up PR to complete the previous one in https://github.com/NOAA-EMC/regional_workflow/pull/537

## CONTRIBUTORS (optional): 
@gsketefian contributed the revision.
epic-cicd-jenkins pushed a commit that referenced this issue Apr 17, 2023
…onf. (#552)

Co-authored-by: Linlin.Pan <Linlin.Pan@noaa.gov>
@climbfuji
Copy link
Collaborator Author

@jkbk2004 @junwang-noaa Is this still needed?

@junwang-noaa
Copy link
Collaborator

@yangfanglin @Qingfu-Liu May I ask if there is some person in physics groupwho are maintaining the csawmg and csawmgt tests?

@yangfanglin
Copy link
Collaborator

Moorthi was the one who were working on CSAW convection and MG MP.
@AnningCheng-NOAA Anning, please take a look and see if you can still provide the support to maintain these RT tests. Otherwise, it is time to remove them

@AnningCheng-NOAA
Copy link
Contributor

AnningCheng-NOAA commented Dec 20, 2023 via email

@climbfuji
Copy link
Collaborator Author

What is the status of this issue?

@AnningCheng-NOAA
Copy link
Contributor

AnningCheng-NOAA commented Feb 1, 2024 via email

@AnningCheng-NOAA
Copy link
Contributor

AnningCheng-NOAA commented Feb 1, 2024 via email

@AnningCheng-NOAA
Copy link
Contributor

AnningCheng-NOAA commented Feb 1, 2024 via email

@climbfuji
Copy link
Collaborator Author

@AnningCheng-NOAA I was just checking on old issues. If you think what is there at the moment is good enough, then we can close this as not planned.

@AnningCheng-NOAA
Copy link
Contributor

Just find the thread and will work on it, It might take some time. An issue that ocurred years ago even when Moorthi was here.

@AnningCheng-NOAA
Copy link
Contributor

AnningCheng-NOAA commented Feb 1, 2024 via email

@AnningCheng-NOAA
Copy link
Contributor

AnningCheng-NOAA commented Feb 12, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants