Hi,
I'm trying to use netcdf-java for reading HDF5 files with radar data.
As simple test program using NetcdfFile.open() was used. I have tried
the sample files found on [ http://www.knmi.nl/opera/odc.html ]. The
files by SMHI fail with this stack trace:
java.lang.IllegalStateException: DataObject doesnt start with OHDR
at ucar.nc2.iosp.hdf5.H5header$DataObject.<init>(H5header.java:1904)
at ucar.nc2.iosp.hdf5.H5header$DataObject.<init>(H5header.java:1818)
at ucar.nc2.iosp.hdf5.H5header.getDataObject(H5header.java:1668)
at ucar.nc2.iosp.hdf5.H5header.access$800(H5header.java:64)
at
ucar.nc2.iosp.hdf5.H5header$DataObjectFacade.<init>(H5header.java:1711)
at ucar.nc2.iosp.hdf5.H5header.readGroupOld(H5header.java:3545)
at ucar.nc2.iosp.hdf5.H5header.access$1000(H5header.java:64)
at ucar.nc2.iosp.hdf5.H5header$H5Group.<init>(H5header.java:1787)
at ucar.nc2.iosp.hdf5.H5header$H5Group.<init>(H5header.java:1756)
at ucar.nc2.iosp.hdf5.H5header.readSuperBlock1(H5header.java:266)
at ucar.nc2.iosp.hdf5.H5header.read(H5header.java:171)
at ucar.nc2.iosp.hdf5.H5iosp.open(H5iosp.java:108)
at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:1333)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:769)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:403)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:373)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:360)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:348)
at HDF5Test.main(HDF5Test.java:16)
Files by other institutions can be read without error messages. HDFView
and h5dump don’t seem to have problems with SMHI’s files. I have tried
netcdf-java 4.0, 4.1, and 4.2.
Do you have more insight into the HDF5 format an can comment on this
issue?
Please find attached the test program used and detailed debug output
for one of the files.
Thanks in advance,
Hermann
import java.io.IOException;
import ucar.nc2.NetcdfFile;
import ucar.nc2.iosp.hdf5.H5iosp;
import ucar.nc2.util.DebugFlags;
public class HDF5Test {
public static void main(String[] args) {
if (args.length != 1) {
System.out.println("Usage: java HDF5Test file.h5");
return;
}
enableFullDebugging();
NetcdfFile dataFile = null;
try {
dataFile = NetcdfFile.open(args[0]);
} catch (Exception e) {
e.printStackTrace(System.err);
} finally {
if (dataFile != null) {
try {
dataFile.close();
} catch (IOException e) {
e.printStackTrace(System.err);
}
}
}
}
private static void enableFullDebugging() {
H5iosp.setDebugFlags(new DebugFlags() {
@Override public void set(String flagName, boolean value) {}
//ignore
@Override public boolean isSet(String flagName) {
return true;
}
});
}
}
H5header 0pened file to
read:'E:/downloads/opera-hd5-samples/SMHI/Z_SCAN_C_ESWI_20100706110000_seang_000000.h5'
size= 17793
versionSB= 1 versionFSS= 0 versionGroup= 0 versionSHMF= 0
sizeOffsets= 4 sizeLengths= 4
isLengthLong= false isOffsetLong= false
btreeLeafNodeSize= 1 btreeInternalNodeSize= 1
fileFlags= 0x0
baseAddress= 0x0
global free space heap Address= 0xffffffffffffffff
eof Address=17793
driver BlockAddress= 0xffffffffffffffff
--> readSymbolTableEntry position=44
nameOffset=0 objectHeaderAddress=76 cacheType=1
Group Entry scratch pad == 116(116) 0(0) 0(0) 0(0) 152(-104) 0(0) 0(0) 0(0)
0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0)
btreeAddress=116 nameHeadAddress=152
<-- end readSymbolTableEntry position=68
--> DataObject.read parsing <root> object ID/address=76
DataObject.read now at position=68 for <root> reposition to 76
version=1 nmess=4
referenceCount=1 headerSize=24
readMessages start at =92 maxMess= 4 maxBytes= 2147483647
--> Message Header starts at =92
-->ObjectHeaderContinuation(16) messageSize=8 flags = 0
--> Message Data starts at=100
Continue offset=256 length=80
count=1 bytesRead=16
---ObjectHeaderContinuation---
readMessages start at =256 maxMess= 3 maxBytes= 80
--> Message Header starts at =256
-->Group(17) messageSize=8 flags = 0
--> Message Data starts at=264
Group btreeAddress=116 nameHeapAddress=152
count=1 bytesRead=16
--> Message Header starts at =272
-->Attribute(12) messageSize=56 flags = 0
--> Message Data starts at=280
*MessageAttribute start pos= 280
MessageAttribute version= 1 flags = 0 nameSize = 16 typeSize=8 spaceSize= 8
name= Conventions
*MessageAttribute before mdt pos= 304
*MessageDatatype start pos= 304
Datatype type=3 version= 1 flags = 0 0 0 byteSize=13 byteOrder=LITTLE
type 3 (String): pad type= 0
*MessageAttribute before mds = 312
*MessageSimpleDataspace start pos= 312
SimpleDataspace version= 1 flags=0 ndims=0 type=0
*MessageAttribute dataPos= 320
count=2 bytesRead=80
---ObjectHeaderContinuation return ---
--> Message Header starts at =108
-->NIL(0) messageSize=0 flags = 0
--> Message Data starts at=116
count=4 bytesRead=24
nmessages read = 4
<--done reading messages for <root>; position=116
<-- end DataObject root
--> GroupOld read <root>
-- readLocalHeap position=152
version=0 size=80 freelistOffset=48 heap starts at dataAddress=176
*now at position=172
-- endLocalHeap position=256
--> GroupBTree read tree at position=116
type=0 level=1 nentries=2
leftAddress=-1 ffffffffffffffff rightAddress=-1 ffffffffffffffff
GroupEntry key=0 address=2564
GroupEntry key=8 address=2528
nonzero node entry at =2564
--> GroupBTree read tree at position=2564
type=0 level=0 nentries=2
leftAddress=-1 ffffffffffffffff rightAddress=2528 9e0
GroupEntry key=0 address=516
GroupEntry key=24 address=2456
nonzero node entry at =2528
--> GroupBTree read tree at position=2528
type=0 level=0 nentries=1
leftAddress=2564 a04 rightAddress=-1 ffffffffffffffff
GroupEntry key=8 address=1460
--Group Node position=516
version=1 nentries=2
--> readSymbolTableEntry position=524
nameOffset=32 objectHeaderAddress=2276 cacheType=0
Group Entry scratch pad == 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0)
0(0) 0(0) 0(0) 0(0) 0(0) 0(0)
<-- end readSymbolTableEntry position=540
--> readSymbolTableEntry position=564
nameOffset=0 objectHeaderAddress=0 cacheType=0
Group Entry scratch pad == 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 17(17) 0(0)
8(8) 0(0) 0(0) 0(0) 0(0) 0(0)
<-- end readSymbolTableEntry position=580
-- Group Node end position=580
--Group Node position=2456
version=1 nentries=1
--> readSymbolTableEntry position=2464
nameOffset=8 objectHeaderAddress=336 cacheType=0
Group Entry scratch pad == 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0)
0(0) 0(0) 0(0) 0(0) 0(0) 0(0)
<-- end readSymbolTableEntry position=2480
-- Group Node end position=2480
--Group Node position=1460
version=1 nentries=1
--> readSymbolTableEntry position=1468
nameOffset=16 objectHeaderAddress=892 cacheType=0
Group Entry scratch pad == 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0)
0(0) 0(0) 0(0) 0(0) 0(0) 0(0)
<-- end readSymbolTableEntry position=1484
-- Group Node end position=1484
Symbol name=dataset1
--> DataObject.read parsing <dataset1> object ID/address=2276
DataObject.read now at position=1484 for <dataset1> reposition to 2276
version=1 nmess=2
referenceCount=1 headerSize=24
readMessages start at =2292 maxMess= 2 maxBytes= 2147483647
--> Message Header starts at =2292
-->Group(17) messageSize=8 flags = 0
--> Message Data starts at=2300
Group btreeAddress=2316 nameHeapAddress=2352
count=1 bytesRead=16
--> Message Header starts at =2308
-->NIL(0) messageSize=0 flags = 0
--> Message Data starts at=2316
count=2 bytesRead=24
nmessages read = 2
<--done reading messages for <dataset1>; position=2316
<-- end DataObject dataset1
Symbol name=
--> DataObject.read parsing <root> object ID/address=0
DataObject.read now at position=2316 for <root> reposition to 0
Memory used file size= 17793
start end size name
0 0 0 header
0 44 44 superblock
44 76 32 SymbolTableEntry
76 116 40 Object root
116 148 32 Group BTree (root)
+ 148 152 4 HOLE
* 152 172 20 Group LocalHeap (root)
+ 172 176 4 HOLE
* 176 256 80 Group LocalHeapData (root)
256 272 16 Message (root) Group(17)
272 336 64 Message (root) Attribute(12)
+ 336 516 180 HOLE
* 516 604 88 Group BtreeNode (root)
* 524 556 32 SymbolTableEntry
+ 556 564 8 HOLE
* 564 596 32 SymbolTableEntry
+ 596 1460 864 HOLE
* 1460 1508 48 Group BtreeNode (root)
* 1468 1500 32 SymbolTableEntry
+ 1500 2276 776 HOLE
* 2276 2316 40 Object dataset1
* 2292 2308 16 Message (dataset1) Group(17)
+ 2308 2456 148 HOLE
* 2456 2504 48 Group BtreeNode (root)
* 2464 2496 32 SymbolTableEntry
+ 2496 2528 32 HOLE
* 2528 2552 24 Group BTree (root)
+ 2552 2564 12 HOLE
* 2564 2596 32 Group BTree (root)
java.lang.IllegalStateException: DataObject doesnt start with OHDR
at ucar.nc2.iosp.hdf5.H5header$DataObject.<init>(H5header.java:1904)
at ucar.nc2.iosp.hdf5.H5header$DataObject.<init>(H5header.java:1818)
at ucar.nc2.iosp.hdf5.H5header.getDataObject(H5header.java:1668)
at ucar.nc2.iosp.hdf5.H5header.access$800(H5header.java:64)
at
ucar.nc2.iosp.hdf5.H5header$DataObjectFacade.<init>(H5header.java:1711)
at ucar.nc2.iosp.hdf5.H5header.readGroupOld(H5header.java:3545)
at ucar.nc2.iosp.hdf5.H5header.access$1000(H5header.java:64)
at ucar.nc2.iosp.hdf5.H5header$H5Group.<init>(H5header.java:1787)
at ucar.nc2.iosp.hdf5.H5header$H5Group.<init>(H5header.java:1756)
at ucar.nc2.iosp.hdf5.H5header.readSuperBlock1(H5header.java:266)
at ucar.nc2.iosp.hdf5.H5header.read(H5header.java:171)
at ucar.nc2.iosp.hdf5.H5iosp.open(H5iosp.java:108)
at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:1333)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:769)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:403)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:373)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:360)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:348)
at HDF5Test.main(HDF5Test.java:16)
