I emailed a couple weeks ago about an error we were seeing when moving
from netcdf 4.4.2.1 to 4.3.1.1 when trying to create an unlimited
time dimension in our output in the GEOS 5 model.
We just tried the new release candidate 4.3.2-rc1 and we were still
seeing the crashing on line 274 of nc4var.c
Unfortunately I still can't reproduce this outside of our model in a
stand alone tester.
I did notice on thing that struck me as peculiar. In my stand alone
tester, when I run with 4.4.2.1 and create a netcdf 4 file with an
unlimited dimension called time and make 1-D variable time using this
dimension an ncdump shows this for the time variable:
int time(time) ;
time:_Storage = "chunked" ;
time:_ChunkSizes = 1 ;
time:_Endianness = "little" ;
with the same code and 4.3.2-rc1 I get this
int time(time) ;
time:_Storage = "chunked" ;
time:_ChunkSizes = 1048576 ;
time:_Endianness = "little" ;
in both cases it says this about the time dimension
time = UNLIMITED ; // (0 currently)
In any case, I am sending the ncdump -sh output for a typical file we
had no problem creating with 4.4.2.1
On 03/05/2014 02:46 PM, Russ Rew wrote:
Hi Ben,
A couple of questions. Your suggestion of setting the the chunking
explicit doesn't seem like it would help as one first has to define the
variable before you can set the chunk size and we are crashing when
defining it unless I am missing something. We did notice that there are
several code changes between 4.2.1 and 4.3.1.1 in the
nc4_find_default_chunksizes2 in nc4var.c which is where we are crashing.
In 4.2.1 if the dimension is unlimited the the chunksize is set to 1 and
it looks like one would skip the code where we are crashing:
suggested_size = (pow((double)DEFAULT_CHUNK_SIZE/(num_values * type_size),
1/(double)(var->ndims - num_set)) * var->dim[d]->len - .5);
In 4.3.1.1 the setting of the unlimited dimension chunksize to 1 was
removed. I'm guessing the code previous to 4.3.1.1 that set the
chunksize to 1 for unlimited dimensions was saving us. We did notice
that the latest version of nc4var.c on github has extra code after the
line in question for set chunksizes of 1-D record variables as well as a
few other changes so I'm wondering if this is a bug fix?
I committed a change to make sure the statement above wouldn't divide by
zero if num_values is zero, but I'm not sure that was the cause of the
crash you encountered.
Unfortunately we have not been able to reproduce this in a small
example program but has there been some change underneath the hood
that that might have occurred that we should be taking a look at?
Could you please capture the output of "ncdump -sh yourfile.nc", for the
file you were able to create using netCDF version 4.2.1, and send it to
support-netcdf@xxxxxxxxxxxxxxxx? With that, we might be able to
reproduce the problem using the ncgen utility.
Thanks.
--Russ
--
Ben Auer, PhD SSAI, Scientific Programmer/Analyst
NASA GSFC, Global Modeling and Assimilation Office
Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
Phone: 301-286-9176 Fax: 301-614-6246
