Thurber, Fred wrote:
I profiled our Grib code (2.2.18) and found a bottleneck in some of the
low-level routines such as Grib1BinaryDataSection.bits2UInt(),
ucar.unidata.io.RandomAccessFile.read(), and, to a lesser degree, in
ucar.grib.GribNumbers.float4().
Has the UCar team ever thought of moving to Java NIO?
we measured NIO and RandomAccessFile when NIO first came out, and there was no
improvement. However, java 1.6 has had lots of improvements, so any new
measurements would be welcome.
I rewrote the
ucar.grib.GribNumbers.float4() method in NIO and got 10 fold performance
boost. I will post the new float4() if anyone is interested.
yes, we'd like to see it.
As far as RandomAccessFile.read(), the reason our program was spending
so much time in it was because of a multiplier inside the
Grib1BinaryDataSection constructor. So 2,558 calls to the
Grib1BinaryDataSection constructor resulted in 273,175,727 calls to
RandomAccessFile.read()! Is there anything I can do about that?
since RandomAccessFile.read() is buffered, these are not actual I/O calls.
Still, there may be improvements in the way Grib1BinaryDataSection works, and
any help you can give would be welcome.
Increase the defaultBufferSize size? It looks as if we have it set to
8K, but I was thinking of increasing that substantially. Can I do this
without a recompile?
Ive done some experiments with different buffer sizes and havent seen a big
improvements with bigger sizes. OTOH, I havent targeted grib files. You can
experiment by using:
NetcdfFile.open(String location, int buffer_size, ucar.nc2.util.CancelTask
cancelTask);
Im not sure if you are using NetcdfFile or the grib library as standalone.
