A couple of questions. Your suggestion of setting the the chunking
explicit doesn't seem like it would help as one first has to define the
variable before you can set the chunk size and we are crashing when
defining it unless I am missing something. We did notice that there are
several code changes between 4.2.1 and 4.3.1.1 in the
nc4_find_default_chunksizes2 in nc4var.c which is where we are crashing.
In 4.2.1 if the dimension is unlimited the the chunksize is set to 1 and
it looks like one would skip the code where we are crashing:
suggested_size = (pow((double)DEFAULT_CHUNK_SIZE/(num_values * type_size),
1/(double)(var->ndims - num_set)) * var->dim[d]->len - .5);
In 4.3.1.1 the setting of the unlimited dimension chunksize to 1 was
removed. I'm guessing the code previous to 4.3.1.1 that set the
chunksize to 1 for unlimited dimensions was saving us. We did notice
that the latest version of nc4var.c on github has extra code after the
line in question for set chunksizes of 1-D record variables as well as a
few other changes so I'm wondering if this is a bug fix?
On Tue, Feb 25, 2014 at 1:59 PM, Ben <Benjamin.M.Auer@xxxxxxxx
<mailto:Benjamin.M.Auer@xxxxxxxx>> wrote:
This is long shot but here it goes.
We recently tried updating netcdf to version 4.3.1.1 from version
4.2.1 in the base libraries in the GEOS model. We found that when
trying to create a diagnostic output file we were getting a crash in
line 288 of nc4var.c which looks like it has something to do with
choosing a chunk size. When it crashed we were in the call to
create the time variable in the file with an nf_def_var call.
Version 4.2.1 and earlier were running fine. The interesting thing
is that the time is an unlimited dimension. When we changed the time
variable to a fixed length dimension 4.3.1.1 ran fine.
Unfortunately we have not been able to reproduce this in a small
example program but has there been some change underneath the hood
that that might have occurred that we should be taking a look at?
There was a discussion on this list a little while back (november?)
about the default chunking for unlimited variables. The default was 1,
which is OK for, say, a 3-d variable with one unlimited dimension but
horrible for a 1-d unlimited variable.
I _think_ there was mention of patching that code, so it may have
broken something. though I can't find that reference anywhere.
Have you tried explicitly setting the chunking on time with it set to
unlimited? You probably want to do that anyway, at least the old default
was really bad for 1-d unlimited variables.
-Chris
--
Ben Auer, PhD SSAI, Scientific Programmer/Analyst
NASA GSFC, Global Modeling and Assimilation Office
Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
Phone: 301-286-9176 Fax: 301-614-6246
