Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GNU hand-rolled with TRAPS: Crash #1

Open
mankoff opened this issue Jul 24, 2023 · 1 comment
Open

GNU hand-rolled with TRAPS: Crash #1

mankoff opened this issue Jul 24, 2023 · 1 comment

Comments

@mankoff
Copy link
Member

mankoff commented Jul 24, 2023

When COMPILE_WITH_TRAPS=YES for the GNU hand-compiled version...

Other runtime compilation flags of interest may be

  • -DNDEBUG for PETSc & IceBin
  • -cold-restart runtime option
Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.

Backtrace for this error:
#0  0x7f9ecb772ad0 in ???
#1  0x7f9ecb771c35 in ???
#2  0x7f9ecb41851f in ???
	at ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
#3  0x7f9ecc20eb12 in _ZN5blitz6DivideIddE5applyEdd
	at /home/kdm/projects/GISS/LIME/opt/include/blitz/ops.h:147
#4  0x7f9ecc20fe77 in _ZN5blitz21_bz_ArrayExprBinaryOpINS_13_bz_ArrayExprINS_21_bz_ArrayExprConstantIdEEEENS1_INS_15FastTV2IteratorIdLi32EEEEENS_6DivideIddEEE10readHelperIdE8fastReadERKS4_RKS7_l
	at /home/kdm/projects/GISS/LIME/opt/include/blitz/array/expr.h:842
#5  0x7f9ecc20f77e in _ZNK5blitz21_bz_ArrayExprBinaryOpINS_13_bz_ArrayExprINS_21_bz_ArrayExprConstantIdEEEENS1_INS_15FastTV2IteratorIdLi32EEEEENS_6DivideIddEEE8fastReadEl
	at /home/kdm/projects/GISS/LIME/opt/include/blitz/array/expr.h:897
#6  0x7f9ecc20ef32 in _ZNK5blitz13_bz_ArrayExprINS_21_bz_ArrayExprBinaryOpINS0_INS_21_bz_ArrayExprConstantIdEEEENS0_INS_15FastTV2IteratorIdLi32EEEEENS_6DivideIddEEEEE8fastReadEl
	at /home/kdm/projects/GISS/LIME/opt/include/blitz/array/expr.h:243
#7  0x7f9ecc20e597 in _ZN5blitz13_tv_evaluatorILb0ELi32EE16evaluate_alignedIdNS_13_bz_ArrayExprINS_21_bz_ArrayExprBinaryOpINS3_INS_21_bz_ArrayExprConstantIdEEEENS3_INS_15FastTV2IteratorIdLi32EEEEENS_6DivideIddEEEEEENS_10_bz_updateIddEEEEvPT_RKT0_T1_
	at /home/kdm/projects/GISS/LIME/opt/include/blitz/tvevaluate.h:105
#8  0x7f9ecc20da79 in _ZN5blitz15chunked_updaterIdNS_13_bz_ArrayExprINS_21_bz_ArrayExprBinaryOpINS1_INS_21_bz_ArrayExprConstantIdEEEENS1_INS_17FastArrayIteratorIdLi1EEEEENS_6DivideIddEEEEEENS_10_bz_updateIddEELi32EE14aligned_updateEPdSC_l
	at /home/kdm/projects/GISS/LIME/opt/include/blitz/globeval.cc:296
#9  0x7f9ecc20ccb8 in _ZN5blitz26_bz_evaluateWithUnitStrideINS_5ArrayIdLi1EEENS_13_bz_ArrayExprINS_21_bz_ArrayExprBinaryOpINS3_INS_21_bz_ArrayExprConstantIdEEEENS3_INS_17FastArrayIteratorIdLi1EEEEENS_6DivideIddEEEEEENS_10_bz_updateIddEEEEvRT_RNSH_10T_iteratorET0_lT1_
	at /home/kdm/projects/GISS/LIME/opt/include/blitz/globeval.cc:475
#10  0x7f9ecc20c51d in _ZN5blitz13_bz_evaluatorILi1EE26evaluateWithStackTraversalINS_5ArrayIdLi1EEENS_13_bz_ArrayExprINS_21_bz_ArrayExprBinaryOpINS5_INS_21_bz_ArrayExprConstantIdEEEENS5_INS_17FastArrayIteratorIdLi1EEEEENS_6DivideIddEEEEEENS_10_bz_updateIddEEEEvRT_T0_T1_
	at /home/kdm/projects/GISS/LIME/opt/include/blitz/globeval.cc:591
#11  0x7f9ecc20bd91 in _ZN5blitz12_bz_evaluateINS_5ArrayIdLi1EEENS_13_bz_ArrayExprINS_21_bz_ArrayExprBinaryOpINS3_INS_21_bz_ArrayExprConstantIdEEEENS3_INS_17FastArrayIteratorIdLi1EEEEENS_6DivideIddEEEEEENS_10_bz_updateIddEEEEvRT_T0_T1_
	at /home/kdm/projects/GISS/LIME/opt/include/blitz/globeval.cc:279
#12  0x7f9ecc20b79a in _ZN5blitz5ArrayIdLi1EEaSINS_13_bz_ArrayExprINS_21_bz_ArrayExprBinaryOpINS3_INS_21_bz_ArrayExprConstantIdEEEENS3_INS_17FastArrayIteratorIdLi1EEEEENS_6DivideIddEEEEEEEERS1_RKNS_6ETBaseIT_EE
	at /home/kdm/projects/GISS/LIME/opt/include/blitz/array/ops.cc:67
#13  0x7f9ecc20aee0 in _ZN5blitz5ArrayIdLi1EEC2INS_21_bz_ArrayExprBinaryOpINS_13_bz_ArrayExprINS_21_bz_ArrayExprConstantIdEEEENS4_INS_17FastArrayIteratorIdLi1EEEEENS_6DivideIddEEEEEENS4_IT_EE
	at /home/kdm/projects/GISS/LIME/opt/include/blitz/array/methods.cc:89
#14  0x7f9ecc25e130 in compute_GpvXAm
	at /home/kdm/projects/GISS/LIME/src/icebin/slib/icebin/modele/GCMRegridder_ModelE.cpp:416
#15  0x7f9ecc272677 in _ZSt13__invoke_implISt10unique_ptrIN6ibmisc6linear14Weighted_EigenESt14default_deleteIS3_EERPFS6_St5arrayIPN8spsparse9SparseSetIliEELm2EERKN6icebin12RegridParamsEPKNSD_6modele19GCMRegridder_ModelEERKN5blitz5ArrayIdLi1EEESP_ccdPKNSD_22RegridMatrices_DynamicEEJSC_SG_RSK_RSN_SX_RcSY_RdRPSQ_EET_St14__invoke_otherOT0_DpOT1_
	at /usr/include/c++/11/bits/invoke.h:61
#16  0x7f9ecc2713c7 in _ZSt8__invokeIRPFSt10unique_ptrIN6ibmisc6linear14Weighted_EigenESt14default_deleteIS3_EESt5arrayIPN8spsparse9SparseSetIliEELm2EERKN6icebin12RegridParamsEPKNSD_6modele19GCMRegridder_ModelEERKN5blitz5ArrayIdLi1EEESP_ccdPKNSD_22RegridMatrices_DynamicEEJSC_SG_RSK_RSN_SX_RcSY_RdRPSQ_EENSt15__invoke_resultIT_JDpT0_EE4typeEOS13_DpOS14_
	at /usr/include/c++/11/bits/invoke.h:97
#17  0x7f9ecc26fcd8 in _ZNSt5_BindIFPFSt10unique_ptrIN6ibmisc6linear14Weighted_EigenESt14default_deleteIS3_EESt5arrayIPN8spsparse9SparseSetIliEELm2EERKN6icebin12RegridParamsEPKNSD_6modele19GCMRegridder_ModelEERKN5blitz5ArrayIdLi1EEESP_ccdPKNSD_22RegridMatrices_DynamicEESt12_PlaceholderILi1EESV_ILi2EESK_SN_SN_ccdPSQ_EE6__callIS6_JOSC_SG_EJLm0ELm1ELm2ELm3ELm4ELm5ELm6ELm7ELm8EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
	at /usr/include/c++/11/functional:422
#18  0x7f9ecc26e78a in _ZNSt5_BindIFPFSt10unique_ptrIN6ibmisc6linear14Weighted_EigenESt14default_deleteIS3_EESt5arrayIPN8spsparse9SparseSetIliEELm2EERKN6icebin12RegridParamsEPKNSD_6modele19GCMRegridder_ModelEERKN5blitz5ArrayIdLi1EEESP_ccdPKNSD_22RegridMatrices_DynamicEESt12_PlaceholderILi1EESV_ILi2EESK_SN_SN_ccdPSQ_EEclIJSC_SG_ES6_EET0_DpOT_
	at /usr/include/c++/11/functional:505
#19  0x7f9ecc26cf0a in _ZSt13__invoke_implISt10unique_ptrIN6ibmisc6linear14Weighted_EigenESt14default_deleteIS3_EERSt5_BindIFPFS6_St5arrayIPN8spsparse9SparseSetIliEELm2EERKN6icebin12RegridParamsEPKNSE_6modele19GCMRegridder_ModelEERKN5blitz5ArrayIdLi1EEESQ_ccdPKNSE_22RegridMatrices_DynamicEESt12_PlaceholderILi1EESW_ILi2EESL_SO_SO_ccdPSR_EEJSD_SH_EET_St14__invoke_otherOT0_DpOT1_
	at /usr/include/c++/11/bits/invoke.h:61
#20  0x7f9ecc26b593 in _ZSt10__invoke_rISt10unique_ptrIN6ibmisc6linear14Weighted_EigenESt14default_deleteIS3_EERSt5_BindIFPFS6_St5arrayIPN8spsparse9SparseSetIliEELm2EERKN6icebin12RegridParamsEPKNSE_6modele19GCMRegridder_ModelEERKN5blitz5ArrayIdLi1EEESQ_ccdPKNSE_22RegridMatrices_DynamicEESt12_PlaceholderILi1EESW_ILi2EESL_SO_SO_ccdPSR_EEJSD_SH_EENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt7is_voidIT_EESt14is_convertibleINSt15__invoke_resultIT0_JDpT1_EE4typeES17_EEE5valueES17_E4typeEOS1C_DpOS1D_
	at /usr/include/c++/11/bits/invoke.h:143
#21  0x7f9ecc268fd5 in _ZNSt17_Function_handlerIFSt10unique_ptrIN6ibmisc6linear14Weighted_EigenESt14default_deleteIS3_EESt5arrayIPN8spsparse9SparseSetIliEELm2EERKN6icebin12RegridParamsEESt5_BindIFPFS6_SC_SG_PKNSD_6modele19GCMRegridder_ModelEERKN5blitz5ArrayIdLi1EEESR_ccdPKNSD_22RegridMatrices_DynamicEESt12_PlaceholderILi1EESX_ILi2EESM_SP_SP_ccdPSS_EEE9_M_invokeERKSt9_Any_dataOSC_SG_
	at /usr/include/c++/11/bits/std_function.h:291
#22  0x7f9ecc1dee92 in ???
#23  0x7f9ecc1d9726 in _ZNK6icebin22RegridMatrices_Dynamic8matrix_dERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt5arrayIPN8spsparse9SparseSetIliEELm2EERKNS_12RegridParamsE
	at /home/kdm/projects/GISS/LIME/src/icebin/slib/icebin/RegridMatrices_Dynamic.cpp:421
#24  0x7f9ecc23d0ad in _ZN6icebin10IceCoupler6coupleESt5arrayIdLm2EERKNS_14VectorMultivecERSt6vectorIS3_SaIS3_EEb
	at /home/kdm/projects/GISS/LIME/src/icebin/slib/icebin/IceCoupler.cpp:495
#25  0x7f9ecc215f21 in _ZN6icebin10GCMCoupler6coupleEdRKNS_14VectorMultivecEb
	at /home/kdm/projects/GISS/LIME/src/icebin/slib/icebin/GCMCoupler.cpp:504
#26  0x7f9ecc27ef83 in _ZN6icebin6modele17GCMCoupler_ModelE6coupleEdRKNS_14VectorMultivecEb
	at /home/kdm/projects/GISS/LIME/src/icebin/slib/icebin/modele/GCMCoupler_ModelE.cpp:1282
#27  0x7f9ecc27ab07 in gcmce_couple_native
	at /home/kdm/projects/GISS/LIME/src/icebin/slib/icebin/modele/GCMCoupler_ModelE.cpp:857
#28  0x7f9ecc279d91 in gcmce_model_start
	at /home/kdm/projects/GISS/LIME/src/icebin/slib/icebin/modele/GCMCoupler_ModelE.cpp:670
#29  0x55c912b4c372 in __lisheeticebin_mod_MOD_cold_start
	at /home/kdm/projects/GISS/LIME/src/modelE/model/LISheetIceBin.F90:555
#30  0x55c912b3726e in __lisnow_mod_MOD_cold_start
	at /home/kdm/projects/GISS/LIME/src/modelE/model/LISnow.F90:429
#31  0x55c91266f8b1 in init_li_
	at /home/kdm/projects/GISS/LIME/src/modelE/model/LANDICE_DRV.f:88
#32  0x55c9123e6847 in input_atm_
	at /home/kdm/projects/GISS/LIME/src/modelE/model/ATM_DRV.f:738
#33  0x55c9123a98c5 in input_
	at /home/kdm/projects/GISS/LIME/src/modelE/model/MODELE.f:1455
#34  0x55c9123ac564 in giss_modele_
	at /home/kdm/projects/GISS/LIME/src/modelE/model/MODELE.f:266
#35  0x55c9123a5178 in modele_maindriver_
	at /home/kdm/projects/GISS/LIME/src/modelE/model/MODELE_DRV.f:27
#36  0x55c9120f1ddc in MAIN__
	at /home/kdm/projects/GISS/LIME/src/modelE/model/main.F90:11
#37  0x55c9120f1e32 in main
	at /home/kdm/projects/GISS/LIME/src/modelE/model/main.F90:13
./r01: 67: exit: Illegal number: -99
 Problem encountered while running 
 >>> INPUT not yet completed <<<

When run in gdb, error is

Thread 1 "r01.exe" received signal SIGFPE, Arithmetic exception.
0x00007ffff7b58b12 in blitz::Divide<double, double>::apply (a=1, b=0) at /home/kdm/projects/GISS/LIME/opt/include/blitz/ops.h:147
147	BZ_DEFINE_BINARY_OP(Divide,/)

A bit more debugging...

(gdb) f 11
#11 0x00007ffff7ba8131 in icebin::modele::compute_GpvXAm (dims=..., paramsA=..., gcmA=0x55555a631520, foceanAOp=..., foceanAOm=..., gridG=73 'I', gridAE=69 'E', eq_rad=6371000, rmO=0x5555679e2350) at /home/kdm/projects/GISS/LIME/src/icebin/slib/icebin/modele/GCMRegridder_ModelE.cpp:416
416	    blitz::Array<double,1> sIpvXOp(1. / IpvXOp->wM);
(gdb) l 416
406	        paramsO.correctA = false;
407	    auto IpvXOp(
408	        rmO->matrix_d(matname,
409	            {dimIp, &dimXOp}, paramsO));
410	
411	    // ----------- Put it all together (XAmvIp)
412	    blitz::Array<double,1> sXOpvIp(1. / XOpvIp->wM);
413	
414	    auto XOmvXAm(XAmvXOm->transpose());
415	    auto &sXOmvXAm(hh.XAmvXOms);
416	    blitz::Array<double,1> sIpvXOp(1. / IpvXOp->wM);
417	    ret->wM.reference(XOpvIp->Mw);
418	    if (paramsA.scale) {
419	        ret->M.reset(new EigenSparseMatrixT(
420	            crop_mvp(dimXOm, dimXOp, 1,
421	                map_eigen_diagonal(sIpvXOp) * *IpvXOp->M) *
422	            map_eigen_diagonal(sXOmvXAm) * XOmvXAm
423	        ));
424	    } else {
425	        ret->M.reset(new EigenSparseMatrixT(
(gdb) p *IpvXOp->wM.data_@1000
$8 = {0 <repeats 703 times>, 399999999.99999994, 400000000, 400000000, 400000000, 0 <repeats 71 times>, 400000000, 400000000, 400000000, 400000000, 400000000, 0, 400000000.00000006, 399999999.99999994, 400000000, 400000000, 400000000, 399999999.99999994, 0 <repeats 65 times>, 400000000, 400000000, 400000000, 400000000, 399999999.99999994, 400000000, 400000000, 400000000, 400000000, 400000000, 400000000, 400000000, 0 <repeats 62 times>, 400000000 <repeats 13 times>, 399999999.99999994, 0 <repeats 57 times>}
@mankoff
Copy link
Member Author

mankoff commented Jul 24, 2023

Same on discover GNU hand-rolled.

@mankoff mankoff mentioned this issue Jul 24, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mankoff