[5.5.4-SNAPSHOT]: Checksum32 throwing checksum exception accessing var in H5 file #1199
Closed
1 task done
Labels
bug
Something isn't working
Versions impacted by the bug
v5.x
What went wrong?
A Panoply user working on a new institutional data product discovered that depending on what version of the app he used, he might not or might be able to extract a variable from an HDF-5 dataset. When there was a failure, a "Checksum invalid" exception was percolating up from
ucar.nc2.filter.Checksum32.java
.On further examination by the user, he found that depending on what version of nccopy he used to create a test dataset, the problem might or might not occur. If he used nccopy 4.7.4, the problem did not occur; if he used nccopy 4.8.1, then the problem occurs. He further noted that in the former case, the H5 dataset was written with superblock version 0, while in the latter it was superblock 2.
In further examination using different versions of Panoply which used differing versions of netcdfAll 5.5.4 snapshots, I found that a copy of Panoply generated in late September last year (using NJ 5.5.3) was able to open the problem dataset and extract the desired variable. A copy generated the end of October (using NJ 5.5.4 snapshot 10/24) was unable to do so.
I note that in between, on Sep. 30 last year, a couple of commits were made to
Checksum32.java
.Following is a stack trace showing where the error occurs. This was generated using an 5.5.4 snapshot just downloaded this afternoon.
Relevant stack trace
Exception doing slice: java.lang.RuntimeException: Checksum invalid java.lang.RuntimeException: Checksum invalid at ucar.nc2.filter.Checksum32.decode(Checksum32.java:82) at ucar.nc2.internal.iosp.hdf5.H5tiledLayoutBB$DataChunk.getByteBuffer(H5tiledLayoutBB.java:235) at ucar.nc2.iosp.LayoutBBTiled.hasNext(LayoutBBTiled.java:101) at ucar.nc2.internal.iosp.hdf5.H5tiledLayoutBB.hasNext(H5tiledLayoutBB.java:143) at ucar.nc2.iosp.IospHelper.readData(IospHelper.java:380) at ucar.nc2.iosp.IospHelper.readDataFill(IospHelper.java:292) at ucar.nc2.internal.iosp.hdf5.H5iospNew.readData(H5iospNew.java:230) at ucar.nc2.internal.iosp.hdf5.H5iospNew.readData(H5iospNew.java:204) at ucar.nc2.NetcdfFile.readData(NetcdfFile.java:2122) at ucar.nc2.Variable.reallyRead(Variable.java:797) at ucar.nc2.Variable._read(Variable.java:736) at ucar.nc2.Variable.read(Variable.java:614) at ucar.nc2.dataset.VariableDS.reallyRead(VariableDS.java:461) at ucar.nc2.dataset.VariableDS._read(VariableDS.java:434) at ucar.nc2.dataset.VariableDS._read(VariableDS.java:444) at ucar.nc2.Variable.read(Variable.java:600) at ucar.nc2.Variable.read(Variable.java:546) at gov.nasa.giss.data.nc.array.NcArray2D.doSlice(NcArray2D.java:417)
Relevant log messages
No response
If you have an example file that you can share, please attach it to this issue.
If so, may we include it in our test datasets to help ensure the bug does not return once fixed?
Note: the test datasets are publicly accessible without restriction.
Yes
Code of Conduct
The text was updated successfully, but these errors were encountered: