Spoke too soon ... it does work for gzipped netcdf files. But it does not
work for grib1 files.
Small .gbx files do show up in the original data directory.
Lak
On Tue, Feb 10, 2009 at 10:30 AM, Valliappa Lakshmanan <
valliappa.lakshmanan@xxxxxxxx> wrote:
> Thanks! Using DiskCache worked.
>
> The default behavior -- of reusing cache files -- does make sense; most of
> your users are reading
> archived data files. It just doesn't work for me because my usage scenario
> is of multiple, simultaneous
> readers that should not step all over each other.
>
> In case someone else needs this, I made this call to DiskCache in an init
> routine:
>
> temporaryDir = File.createTempFile("wnc", "dir");
> temporaryDir.delete();
> temporaryDir.mkdir();
> temporaryDir.deleteOnExit();
> log.info("Using " + temporaryDir + " to store temporary netcdf
> files");
> DiskCache.setRootDirectory(temporaryDir.getAbsolutePath());
> DiskCache.setCachePolicy(true); // always in cache
>
> and periodically:
> if (temporaryDir != null){
> for (File f : temporaryDir.listFiles()){
> f.delete();
> }
> }
> DiskCache also has routines for cleaning things up based on
> least-recently-used; but there are temporaries,
> so it may be simpler to just get rid of all of them.
>
> Lak
>
>
>
> On Mon, Feb 9, 2009 at 2:21 PM, John Caron <caron@xxxxxxxxxxxxxxxx> wrote:
>
>> Hi Lak:
>>
>> This behavior is controlled by the singleton class
>> ucar.nc2.util.DiskCache. See the javadocs for that:
>>
>> http://www.unidata.ucar.edu/software/netcdf-java/v4.0/javadoc/index.html
>>
>> An overview is at:
>>
>>
>> http://www.unidata.ucar.edu/software/netcdf-java/tutorial/NetcdfFile.html#DiskCache
>>
>> see embedded comments below:
>>
>> Valliappa Lakshmanan wrote:
>> > I've been using the Java Netcdf API and am running into several issues
>> > with the temporary files that are created.
>> >
>> > For example, when a gzipped NetCDF file is read, an uncompressed
>> > version is written to disk before it is read. I assume that this is
>> > because the NetCDF API allows for file seeks etc.
>> > which wouldn't be possible if the file remained gzipped. But (and this
>> > is the problem), the uncompressed
>> > file is stored in the SAME directory as the original file. This causes
>> > three major problems (in increasing order
>> > of severity):
>> > (a) The data directory (which would be considered read-only) is being
>> > modified. This causes problems because of the I/O optimizations that we
>> do on real-time systems
>>
>> you can make all temp files go to a different directory.
>>
>> > (b) The temporary file is not automatically cleaned up, so these
>> > temporaries start to fill up the disk.
>>
>> see overview for example on how to cleanup.
>>
>> > But removing the temporary file when the original file is closed
>> > is not enough because of problem (c).
>> > (c) if there are two simultaneous programs reading a gzipped file,
>> > then a potential for race conditions exists
>>
>> yes, this is a good point. the default case is optimized for
>> single-threaded use. The TDS uses a
>> different strategy for this reason. Do you actually see this problem? If
>> so, in what context?
>>
>> >
>> > The same problem seems to exist when reading a Grib1 file (a .gbx
>> > temporary file is created).
>> >
>> > Is there any work-around for this problem?
>> >
>> > Lak
>> >
>> > p.s. I suggest the consistent use of java.io.File's createTempFile()
>> ...
>>
>> That might be a useful option. Do you actually need it, or is this
>> theoretical?
>>
>> The default case is to avoid the overhead of unzipping/indexing more than
>> once.
>>
>>
>>
>> >
>> >
>> > ------------------------------------------------------------------------
>> >
>> > _______________________________________________
>> > netcdfgroup mailing list
>> > netcdfgroup@xxxxxxxxxxxxxxxx
>> > For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
>
