[decoders] Large GRIB files

• To: decoders@xxxxxxxxxxxxxxxx
• Subject: [decoders] Large GRIB files
• From: Ederson Ferreira <ederson.jau@xxxxxxxxx>
• Date: Wed, 16 Nov 2011 15:15:31 -0200

Hi,

I'm trying to read a large GRIB file using the java decoder, but I'm
getting the following exception:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
 at java.util.Arrays.copyOf(Arrays.java:2882)
at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
 at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572)
 at java.lang.StringBuffer.append(StringBuffer.java:320)
at ucar.grib.grib1.Grib1Input.seekHeader(Grib1Input.java:247)
 at ucar.grib.grib1.Grib1Input.scan(Grib1Input.java:129)

Since the grib file has 3.8Gb, the exception is thrown when the class tries
to load the whole file to the memory.

Is there a way to seek the grib headers in-disk?

I'm a newbie in the grib file format, but looking at the source code for
GribInput, the seekHeader method loads a bunch of bytes into the memory and
then look for the header pattern. Is this load really necessary?

Att,

-- 
Ederson Ferreira de Jesus
Matemático Aplicado - UNICAMP
"Time is an illusion. Lunchtime doubly so."
Adams, Douglas