Hi Peter:
it looks like a bug, im looking at it now. thanks for the nice example.
Peter.Hollemans@xxxxxxxx wrote:
I think I may have found a bug in 4.1, and I'd like to know if it's actually a bug, and if there's
a workaround.
What I'm trying to do is write a NetCDF file by alternating define mode and writing mode. It's
difficult in my code to get around doing that, because there are some preliminary CF
metadata variables to set up before writing actual variable data, and those code modules are
independent of each other.
What I decided to do was initially allocate 64k in the header right after creating the file
object, so that it has enough space for the global metadata and then when I add new
variables and attributes, it doesn't have to rewrite the whole file as the documentation states
can be I/O intensive. But when I alternately define new variables and write data to them, the
variable data in never written in some cases.
Here's a sample program that reproduces the problem:
----------------------
import ucar.nc2.*;
import ucar.ma2.*;
public class WriteTest {
public static void main (String args[]) throws Exception {
NetcdfFileWriteable ncFile = NetcdfFileWriteable.createNew (args[0], false);
ncFile.setExtraHeaderBytes (64*1024);
Dimension dim = ncFile.addDimension ("time", 100);
double[] jackData = new double[100];
for (int i = 0; i < 100; i++) jackData[i] = i;
double[] jillData = new double[100];
for (int i = 0; i < 100; i++) jillData[i] = i;
Dimension[] dims = new Dimension[] {dim};
ncFile.addVariable ("jack", DataType.DOUBLE, dims);
ncFile.addVariableAttribute ("jack", "where", "up the hill");
ncFile.create();
int[] start = new int[] {0};
int[] count = new int[] {100};
ncFile.write ("jack", start, Array.factory (double.class, count, jackData));
ncFile.setRedefineMode (true);
ncFile.addVariable ("jill", DataType.DOUBLE, dims);
ncFile.addVariableAttribute ("jill", "where", "up the hill");
ncFile.setRedefineMode (false);
ncFile.write ("jill", start, Array.factory (double.class, count, jillData));
ncFile.flush();
ncFile.close();
} // main
} // WriteTest class
----------------------
I compile and run it like this:
----------------------
phollema@Bean<cwf-3.2.4> javac -cp lib/toolsUI-4.1.jar:. WriteTest.java
phollema@Bean<cwf-3.2.4> java -cp lib/toolsUI-4.1.jar:. WriteTest mytest.nc
phollema@Bean<cwf-3.2.4> ncdump mytest.nc
netcdf mytest {
dimensions:
time = 100 ;
variables:
double jack(time) ;
jack:where = "up the hill" ;
double jill(time) ;
jill:where = "up the hill" ;
data:
jack = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37,
38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,
56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73,
74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91,
92, 93, 94, 95, 96, 97, 98, 99 ;
jill = 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0 ;
}
----------------------
As you can see, it never wrote the data for "jill". I found that if I remove the call to
setExtraHeaderBytes(), it works correctly and writes all the data. But that defeats my ability
to ensure that the entire file won't be re-written when I want to add a new variable. I want to
be able to write files with hundreds of megabytes of data, and never have it re-write the data
when I go into and out of define mode.
Any suggestions?
Peter
_______________________________________________
netcdf-java mailing list
netcdf-java@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
