Re: [netcdf-java] [netcdfgroup] Java API: problems with temporary files

Hi Lak:

This behavior is controlled by the singleton class ucar.nc2.util.DiskCache. See 
the javadocs for that:

  http://www.unidata.ucar.edu/software/netcdf-java/v4.0/javadoc/index.html

An overview is at:

  
http://www.unidata.ucar.edu/software/netcdf-java/tutorial/NetcdfFile.html#DiskCache

see embedded comments below:

Valliappa Lakshmanan wrote:
> I've been using the Java Netcdf API and am running into several issues
> with the temporary files that are created.
> 
> For example, when a gzipped NetCDF file is read, an uncompressed
> version is written to disk before it is read.  I assume that this is
> because the NetCDF API allows for file seeks etc.
> which wouldn't be possible if the file remained gzipped.   But (and this
> is the problem), the uncompressed
> file is stored in the SAME directory as the original file.  This causes
> three major problems (in increasing order
> of severity):
>    (a) The data directory (which would be considered read-only) is being
> modified.  This causes problems because of the I/O optimizations that we do 
> on real-time systems

you can make all temp files go to a different directory.

>    (b) The temporary file is not automatically cleaned up, so these
> temporaries start to fill up the disk.

see overview for example on how to cleanup.

>         But removing the temporary file when the original file is closed
> is not enough because of problem (c).
>    (c) if there are two simultaneous programs reading a gzipped file,
> then a potential for race conditions exists

yes, this is a good point. the default case is optimized for single-threaded 
use. The TDS uses a
different strategy for this reason. Do you actually see this problem? If so, in 
what context?

> 
> The same problem seems to exist when reading a Grib1 file (a .gbx
> temporary file is created).
> 
> Is there any work-around for this problem?
> 
> Lak
> 
> p.s.  I suggest the consistent use of java.io.File's createTempFile() ...

That might be a useful option. Do you actually need it, or is this theoretical?

The default case is to avoid the overhead of unzipping/indexing more than once.



> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/ 


  • 2009 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: