[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #FFE-268601]: Fwd: Fix for GRIB File Descriptor Leak

Hi Aaron:

Yes, I think you are right that there is a file leak here. I have to look 
closely at your changes to see if there is any collateral problems.

In the future, the easiest thing for us is to issue a pull request against that 
branch on gihub.

John 


> My apologies, I may have sent this to the wrong list.
> 
> Aaron
> 
> -------- Original Message --------
> Subject:      Fix for GRIB File Descriptor Leak
> Date:         Fri, 16 May 2014 15:22:05 -0600
> From:         Aaron Braeckel <address@hidden>
> Reply-To:     address@hidden
> Organization:         National Center for Atmospheric Research / RAL
> To:   address@hidden
> CC:   Rob Weingruber <address@hidden>
> 
> 
> 
> Hello,
> 
> Rob (CCed) emailed in February (subject: "EOFException: Reading
> file.grb2.ncx at 0 file length = 0") about a file descriptor leak
> observed with the Java netCDF when reading invalid GRIB index files.  I
> was able to reliably replicate the file leak locally by manually setting
> the index file to 0 size before opening a GRIB file.  I observed the
> file leak through the set of open file descriptors (through both the
> Java UnixOperatingSystemMXBean class and the lsof command).
> 
> The attached source files are a patch for this issue against the
> edu.ucar:grib:4.3.21 source code.  This patch is quite simple, below I
> describe the individual changes.  Would it be possible to apply these
> changes and make a new release?  If so, when would the release be made?
> We are currently maintaining our own branch of the GRIB library because
> we need this fix in operations, which is undesirable for a number of
> reasons.  I can provide a main() method that demonstrates this problem
> if it would be of interest.
> 
> Thanks!
> Aaron and Rob
> 
> ===============================
> 
> The source of the problem was in Grib1CollectionBuilder and
> Grib2CollectionBuilder (the code changes to each are identical):
> 
> In /readIndex/(String filename) a new RandomAccessFile was being created
> but was never closed.  This fix definitively resolved our file leak
> issue, and logically the created RandomAccessFile must be closed in this
> method since it is only used there.
> 
> <     return readIndex( new RandomAccessFile(filename, "r") );
> ---
> >     RandomAccessFile raf = new RandomAccessFile( filename, "r" );
> >     boolean success = readIndex( raf );
> >     raf.close();
> >     return success;
> 
> 
> In /readOrCreateIndex/() there is more code that opens a
> RandomAccessFile but may not close it.  This code did not resolve our
> issue, but is similar to the fix above and may resolve a file leak in
> other situations.  In this case I note that gc.setIndexRaf(indexRaf) is
> called just before the changed code segment, and therefore the RAF may
> be closed properly without this change.  I don't know the code and flow
> of operations well enough to be sure.
> 
> <       readIndex(indexRaf);
> ---
> >       boolean success = readIndex(indexRaf);
> >       if( !success ){
> >         indexRaf.close();
> >       }
> 
> 
> 
> 
> 
> 
> 

Ticket Details
===================
Ticket ID: FFE-268601
Department: Support netCDF Java
Priority: Normal
Status: Open