Update hdf5, netcdf-c, netcdf-fortran #453
Assignees
Comments
climbfuji
changed the title
|
Update: netcdf-c 4.9.1 currently doesn't compile with hdf5 1.14.0, but netcdf-c 4.9.0 does. This is the problem: Unidata/netcdf-c#2614 Let's hold off on updating to 4.9.1 until the netcdf-c developers fixed this. |
|
I don't think you should hold off. The problem is with byterange code, and we never use it. In fact, we need to build with --disable-byterange anyway, so that disables the code with the bug. Meanwhile 4.9.1 contains some important fixes for the new quantize and zstd compression features, which the UFS teams wants to try out, and which will also benefit anyone at NCAR who's trying to use these new compression features. |
|
@AlexanderRichert-NOAA See here. I don't agree, we probably want to build with byterange as soon as the fix has been released (4.9.2) on all platforms EXCEPT the FISMA high systems where we turn it off, together with other things. |
|
Our tests based on the netcdf/4.9.1 with two options:
--disable-byterange
--disable-libxml2
The UFS failed.
…On Mon, Feb 13, 2023 at 9:10 AM Dom Heinzeller ***@***.***> wrote:
@AlexanderRichert-NOAA <https://github.com/AlexanderRichert-NOAA> See
here. I don't agree, we probably want to build with byterange as soon as
the fix has been released (4.9.2) on all platforms EXCEPT the FISMA high
systems where we turn it off, together with other things.
—
Reply to this email directly, view it on GitHub
<#453 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKWSMFH5EFCGFRHYF57H3GDWXI6FPANCNFSM6AAAAAAUD6O44I>
.
You are receiving this because you were assigned.Message ID:
***@***.***>
|
|
What failed? Did it crash? Or produce different results?
… On Feb 13, 2023, at 7:15 AM, Hang-Lei-NOAA ***@***.***> wrote:
Our tests based on the netcdf/4.9.1 with two options:
--disable-byterange
--disable-libxml2
The UFS failed.
On Mon, Feb 13, 2023 at 9:10 AM Dom Heinzeller ***@***.***>
wrote:
> @AlexanderRichert-NOAA <https://github.com/AlexanderRichert-NOAA> See
> here. I don't agree, we probably want to build with byterange as soon as
> the fix has been released (4.9.2) on all platforms EXCEPT the FISMA high
> systems where we turn it off, together with other things.
>
> —
> Reply to this email directly, view it on GitHub
> <#453 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AKWSMFH5EFCGFRHYF57H3GDWXI6FPANCNFSM6AAAAAAUD6O44I>
> .
> You are receiving this because you were assigned.Message ID:
> ***@***.***>
>
—
Reply to this email directly, view it on GitHub <#453 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5C2RO5H5X4X4BABBKSF5LWXI6Y5ANCNFSM6AAAAAAUD6O44I>.
You are receiving this because you were assigned.
|
|
UFS model crashed with segmentation error when using netcdf.
On Mon, Feb 13, 2023 at 9:55 AM Dom Heinzeller ***@***.***>
wrote:
… What failed? Did it crash? Or produce different results?
> On Feb 13, 2023, at 7:15 AM, Hang-Lei-NOAA ***@***.***> wrote:
>
>
> Our tests based on the netcdf/4.9.1 with two options:
> --disable-byterange
> --disable-libxml2
> The UFS failed.
>
> On Mon, Feb 13, 2023 at 9:10 AM Dom Heinzeller ***@***.***>
> wrote:
>
> > @AlexanderRichert-NOAA <https://github.com/AlexanderRichert-NOAA> See
> > here. I don't agree, we probably want to build with byterange as soon
as
> > the fix has been released (4.9.2) on all platforms EXCEPT the FISMA
high
> > systems where we turn it off, together with other things.
> >
> > —
> > Reply to this email directly, view it on GitHub
> > <
#453 (comment)
>,
> > or unsubscribe
> > <
https://github.com/notifications/unsubscribe-auth/AKWSMFH5EFCGFRHYF57H3GDWXI6FPANCNFSM6AAAAAAUD6O44I
>
> > .
> > You are receiving this because you were assigned.Message ID:
> > ***@***.***>
> >
> —
> Reply to this email directly, view it on GitHub <
#453 (comment)>,
or unsubscribe <
https://github.com/notifications/unsubscribe-auth/AB5C2RO5H5X4X4BABBKSF5LWXI6Y5ANCNFSM6AAAAAAUD6O44I
>.
> You are receiving this because you were assigned.
>
—
Reply to this email directly, view it on GitHub
<#453 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKWSMFBT3YPLYOJNQZ7KBCTWXJDO3ANCNFSM6AAAAAAUD6O44I>
.
You are receiving this because you were assigned.Message ID:
***@***.***>
|
|
Can we have more detail about the failure posted here? |
|
In the meanwhile, I am going to revert netcdf-c back to 4.9.0 in the first unified-dev PR (#455) or I will never be able to complete that step. |
|
@edward Hartnett - NOAA Affiliate ***@***.***> the failure
docs was kept on cactus. It has been switched to operation. I will redo a
test on Acorn for that.
…On Mon, Feb 13, 2023 at 10:22 AM Dom Heinzeller ***@***.***> wrote:
In the meanwhile, I am going to revert netcdf-c back to 4.9.0 in the first
unified-dev PR (#455 <#455>)
or I will never be able to complete that step.
—
Reply to this email directly, view it on GitHub
<#453 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKWSMFFR3BLMQTSW25BZFADWXJGT3ANCNFSM6AAAAAAUD6O44I>
.
You are receiving this because you were assigned.Message ID:
***@***.***>
|
|
Failure provided by Pan Li on Acorn when using hdf 1.14.0 and netcdf 4.9.1:
=============================
***@***.***:~> vi
/lfs/h1/emc/ptmp/li.pan/FV3_RT/rt_192415/cpld_control_p8/err
fv3.exe 000000000042FCFB MAIN__ 381 UFS.F90
fv3.exe 000000000042EC92 Unknown Unknown Unknown
libc-2.31.so 00001553132852BD __libc_start_main Unknown Unknown
fv3.exe 000000000042EBAA Unknown Unknown Unknown
C
fv3.exe 00000000013D2DAF _ZN5ESMCI3VMK5ent 2320
ESMCI_VMKernel.C
fv3.exe 00000000013BC859 _ZN5ESMCI2VM5ente 1216
ESMCI_VM.C
fv3.exe 0000000000DEB857 c_esmc_ftablecall 981
ESMCI_FTable.C
fv3.exe 00000000009D2700 esmf_compmod_mp_e 1223
ESMF_Comp.F90
fv3.exe 00000000007D3A96 esmf_gridcompmod_ 1412
ESMF_GridComp.F90
fv3.exe 000000000042FCFB MAIN__ 381 UFS.F90
fv3.exe 000000000042EC92 Unknown Unknown Unknown
libc-2.31.so 0000154AF9E6E2BD __libc_start_main Unknown Unknown
fv3.exe 000000000042EBAA Unknown Unknown Unknown
xe 0000000000DEE04A ESMCI_FTableCallE 824
ESMCI_FTable.C
fv3.exe 00000000013D2DAF _ZN5ESMCI3VMK5ent 2320
ESMCI_VMKernel.C
fv3.exe 00000000013BC859 _ZN5ESMCI2VM5ente 1216
ESMCI_VM.C
fv3.exe 0000000000DEB857 c_esmc_ftablecall 981
ESMCI_FTable.C
fv3.exe 00000000009D2700 esmf_compmod_mp_e 1223
ESMF_Comp.F90
fv3.exe 00000000007D3A96 esmf_gridcompmod_ 1412
ESMF_GridComp.F90
fv3.exe 000000000042FCFB MAIN__ 381 UFS.F90
fv3.exe 000000000042EC92 Unknown Unknown Unknown
libc-2.31.so 00001507FBB082BD __libc_start_main Unknown Unknown
fv3.exe 000000000042EBAA Unknown Unknown Unknown
C
fv3.exe 00000000013D2DAF _ZN5ESMCI3VMK5ent 2320
ESMCI_VMKernel.C
fv3.exe 00000000013BC859 _ZN5ESMCI2VM5ente 1216
ESMCI_VM.C
fv3.exe 0000000000DEB857 c_esmc_ftablecall 981
ESMCI_FTable.C
fv3.exe 00000000009D2700 esmf_compmod_mp_e 1223
ESMF_Comp.F90
fv3.exe 00000000007D3A96 esmf_gridcompmod_ 1412
ESMF_GridComp.F90
fv3.exe 000000000042FCFB MAIN__ 381 UFS.F90
fv3.exe 000000000042EC92 Unknown Unknown Unknown
libc-2.31.so 000014D0BAB752BD __libc_start_main Unknown Unknown
================================================================
On Mon, Feb 13, 2023 at 10:41 AM Hang Lei - NOAA Affiliate <
***@***.***> wrote:
… @edward Hartnett - NOAA Affiliate ***@***.***> the failure
docs was kept on cactus. It has been switched to operation. I will redo a
test on Acorn for that.
On Mon, Feb 13, 2023 at 10:22 AM Dom Heinzeller ***@***.***>
wrote:
> In the meanwhile, I am going to revert netcdf-c back to 4.9.0 in the
> first unified-dev PR (#455
> <#455>) or I will never be
> able to complete that step.
>
> —
> Reply to this email directly, view it on GitHub
> <#453 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AKWSMFFR3BLMQTSW25BZFADWXJGT3ANCNFSM6AAAAAAUD6O44I>
> .
> You are receiving this because you were assigned.Message ID:
> ***@***.***>
>
|
|
Update from cactus:
*/lfs/h2/emc/ptmp/li.pan/FV3_RT/rt_219124*
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
fv3.exe 0000000005EF5BEA Unknown Unknown Unknown
libpthread-2.31.s 0000149272F148C0 Unknown Unknown Unknown
libnetcdff.so.7.1 0000149277AA3C5F netcdf_mp_nf90_in Unknown Unknown
fv3.exe 00000000059926D4 netcdf_io_mod_mp_ 1511
netcdf_io.F90
fv3.exe 0000000005990A84 netcdf_io_mod_mp_ 85
netcdf_read_data.inc
fv3.exe 00000000059ABFAC netcdf_io_mod_mp_ 44
compressed_read.inc
fv3.exe 0000000005A02F71 grid2_mod_mp_open 195
grid2.F90
fv3.exe 0000000005A04276 grid2_mod_mp_open 253
grid2.F90
fv3.exe 0000000005A0442D grid2_mod_mp_grid 144
grid2.F90
fv3.exe 0000000005933C83 fms_mod_mp_fms_in 428 fms.F90
fv3.exe 0000000004313BD4 mom_cap_mod_mp_in 465
mom_cap.F90
fv3.exe 0000000000DEA414 _ZN5ESMCI6FTable1 2167
ESMCI_FTable.C
fv3.exe 0000000000DEE04A ESMCI_FTableCallE 824
ESMCI_FTable.C
fv3.exe 00000000013D2F9B _ZN5ESMCI3VMK5ent 1124
ESMCI_VMKernel.C
fv3.exe 00000000013BC859 _ZN5ESMCI2VM5ente 1216
ESMCI_VM.C
fv3.exe 0000000000DEB857 c_esmc_ftablecall 981
ESMCI_FTable.C
fv3.exe 00000000009D2700 esmf_compmod_mp_e 1223
ESMF_Comp.F90
fv3.exe 00000000007D3A96 esmf_gridcompmod_ 1412
ESMF_GridComp.F90
fv3.exe 000000000045DC7F nuopc_driver_mp_l 2713
NUOPC_Driver.F90
fv3.exe 0000000000484C2A nuopc_driver_mp_i 1313
NUOPC_Driver.F90
fv3.exe 000000000048E0F3 nuopc_driver_mp_i 476
NUOPC_Driver.F90
fv3.exe 0000000000DEA414 _ZN5ESMCI6FTable1 2167
ESMCI_FTable.C
fv3.exe 0000000000DEE04A ESMCI_FTableCallE 824
ESMCI_FTable.C
fv3.exe 00000000013D2DAF _ZN5ESMCI3VMK5ent 2320
ESMCI_VMKernel.C
fv3.exe 00000000013BC859 _ZN5ESMCI2VM5ente 1216
ESMCI_VM.C
fv3.exe 0000000000DEB857 c_esmc_ftablecall 981
ESMCI_FTable.C
fv3.exe 00000000009D2700 esmf_compmod_mp_e 1223
ESMF_Comp.F90
fv3.exe 00000000007D3A96 esmf_gridcompmod_ 1412
ESMF_GridComp.F90
fv3.exe 000000000042FCFB MAIN__ 381 UFS.F90
fv3.exe 000000000042EC92 Unknown Unknown Unknown
libc-2.31.so 00001492729252BD __libc_start_main Unknown Unknown
fv3.exe 000000000042EBAA Unknown Unknown Unknown
On Mon, Feb 13, 2023 at 11:25 AM Hang Lei - NOAA Affiliate <
***@***.***> wrote:
… Failure provided by Pan Li on Acorn when using hdf 1.14.0 and netcdf 4.9.1:
=============================
***@***.***:~> vi
/lfs/h1/emc/ptmp/li.pan/FV3_RT/rt_192415/cpld_control_p8/err
fv3.exe 000000000042FCFB MAIN__ 381 UFS.F90
fv3.exe 000000000042EC92 Unknown Unknown Unknown
libc-2.31.so 00001553132852BD __libc_start_main Unknown
Unknown
fv3.exe 000000000042EBAA Unknown Unknown Unknown
C
fv3.exe 00000000013D2DAF _ZN5ESMCI3VMK5ent 2320
ESMCI_VMKernel.C
fv3.exe 00000000013BC859 _ZN5ESMCI2VM5ente 1216
ESMCI_VM.C
fv3.exe 0000000000DEB857 c_esmc_ftablecall 981
ESMCI_FTable.C
fv3.exe 00000000009D2700 esmf_compmod_mp_e 1223
ESMF_Comp.F90
fv3.exe 00000000007D3A96 esmf_gridcompmod_ 1412
ESMF_GridComp.F90
fv3.exe 000000000042FCFB MAIN__ 381 UFS.F90
fv3.exe 000000000042EC92 Unknown Unknown Unknown
libc-2.31.so 0000154AF9E6E2BD __libc_start_main Unknown
Unknown
fv3.exe 000000000042EBAA Unknown Unknown Unknown
xe 0000000000DEE04A ESMCI_FTableCallE 824
ESMCI_FTable.C
fv3.exe 00000000013D2DAF _ZN5ESMCI3VMK5ent 2320
ESMCI_VMKernel.C
fv3.exe 00000000013BC859 _ZN5ESMCI2VM5ente 1216
ESMCI_VM.C
fv3.exe 0000000000DEB857 c_esmc_ftablecall 981
ESMCI_FTable.C
fv3.exe 00000000009D2700 esmf_compmod_mp_e 1223
ESMF_Comp.F90
fv3.exe 00000000007D3A96 esmf_gridcompmod_ 1412
ESMF_GridComp.F90
fv3.exe 000000000042FCFB MAIN__ 381 UFS.F90
fv3.exe 000000000042EC92 Unknown Unknown Unknown
libc-2.31.so 00001507FBB082BD __libc_start_main Unknown
Unknown
fv3.exe 000000000042EBAA Unknown Unknown Unknown
C
fv3.exe 00000000013D2DAF _ZN5ESMCI3VMK5ent 2320
ESMCI_VMKernel.C
fv3.exe 00000000013BC859 _ZN5ESMCI2VM5ente 1216
ESMCI_VM.C
fv3.exe 0000000000DEB857 c_esmc_ftablecall 981
ESMCI_FTable.C
fv3.exe 00000000009D2700 esmf_compmod_mp_e 1223
ESMF_Comp.F90
fv3.exe 00000000007D3A96 esmf_gridcompmod_ 1412
ESMF_GridComp.F90
fv3.exe 000000000042FCFB MAIN__ 381 UFS.F90
fv3.exe 000000000042EC92 Unknown Unknown Unknown
libc-2.31.so 000014D0BAB752BD __libc_start_main Unknown
Unknown
================================================================
On Mon, Feb 13, 2023 at 10:41 AM Hang Lei - NOAA Affiliate <
***@***.***> wrote:
> @edward Hartnett - NOAA Affiliate ***@***.***> the failure
> docs was kept on cactus. It has been switched to operation. I will redo a
> test on Acorn for that.
>
> On Mon, Feb 13, 2023 at 10:22 AM Dom Heinzeller ***@***.***>
> wrote:
>
>> In the meanwhile, I am going to revert netcdf-c back to 4.9.0 in the
>> first unified-dev PR (#455
>> <#455>) or I will never
>> be able to complete that step.
>>
>> —
>> Reply to this email directly, view it on GitHub
>> <#453 (comment)>,
>> or unsubscribe
>> <https://github.com/notifications/unsubscribe-auth/AKWSMFFR3BLMQTSW25BZFADWXJGT3ANCNFSM6AAAAAAUD6O44I>
>> .
>> You are receiving this because you were assigned.Message ID:
>> ***@***.***>
>>
>
|
|
Hello @Hang-Lei-NOAA ! I think you meant @edwardhartnett but thanks for making me aware of all the cool stuff NOAA is up to on GitHub! I was involved with getting the Canadian feds into using GitHub so am happy to see more like it. |
climbfuji
changed the title
|
This update was completed in #454 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Please describe the package or library you would like to add to spack-stack.
hdf5 1.14.0 was released, we should test this version and switch if we can. See also NOAA-EMC/hpc-stack#513.
For netCDF: netcdf-c is now at 4.9.0 (4.9.1 will be release shortly, should we wait for this one?), netcdf-fortran at 4.6.0, netcdf-cxx4 is at 4.3.1 (but I think we don't want to use netcdf-cxx4 anyway).
What applications will be using this package or library?
All
Is there already a package or library in spack-stack that provides this, or related, functionality?
Earlier versions are used everywhere.
Additional context
n/a
Will This Package be Needed in a NOAA Operational Application?
Those libraries have all been approved on WCOSS2.
The text was updated successfully, but these errors were encountered: