Ed Hartnett wrote:
Jeff Whitaker <jswhit@xxxxxxxxxxx> writes:
Ed Hartnett wrote:
Howdy Jeff,
Just an update: we have confirmed this bug and I am working on it
now. I have also added this to our automatic testing, so that we can
see it break here on our test machines, and ensure that once it's
fixed, it doesn't break again.
I will let you know when this fix is available in the snapshot -
hopefully not more than a few days from now...
Thanks again for find this!
Ed
Ed: Any progress on this one?
Howdy Jeff!
Yes, there has been much progress, and it you get the snapshot I believe
your case will now work. (And if you look in the ncdump and libsrc4
directories you can see where I use your CDL and the files generated
from it for a new series of cross-platform testing that's now part of
the build.)
However I am not done with this problem, there are some other cases that
need to be checked (like compound inside of vlen, and vice-versa). So I
am still working on the code and adding more cross-platform tests. I
have also taken this opportunity for some core libsrc4 refactoring with
respect to the handling of user-defined types.
I don't think I have yet solved all the compound types problems on all
platforms, as my new tests are still causing failures on the IRIX and
AIX platforms. I am looking into this now.
Ed: Great, thanks - I'll give the latest snapshot a try.
Also, to answer your original concern, this does not mean that the data
files are not portable. They are. The bug was in the way that the
netCDF-4 gave out information on the user-defined types in the
file. This is why generic read programs (like ncdump) can demonstrate
the problem, while cross-platform testing of the compound type do not
show the problem.
As I understand it, netcdf-4.0.1 reads compound types assuming that the
offsets conform to the default padding of the compiler, regardless of
what offsets were actually used. So, if you don't use the default
alignment when you write the file, you won't be able to read it with
netcdf 4 lib. Or, if you do use the default compiler alignment you
can't read it on another system with a different alignment. The example
program I sent April 29 demonstrates this (if the file is written with
'packed alignment', it can't be read with ncdump).
This doesn't only apply to 'generic' programs like ncdump. User code
suffers as well, because the information returned about the offsets is
wrong. I guess you mean the data inside the file is correct, you just
can't read it with the netcdf4 library (but you could with a generic
HDF5 program). If that's what you mean, I agree. However, if I can't
read the file on another platform with the same library that wrote it,
that sounds like the definition of "non-portable" to me. Whether the
non-portablity is is in the file or the library is semantics.
But the files that you created are perfectly fine, and when this bug fix
is complete they will be understood properly by ncdump and other
programs that generically read a netCDF-4 file with compounds inside
compound types.
Great.
As always, a good way to confirm this is to look at the h5dump
output. If h5dump shows the data properly, then it is stored properly in
the HDF5 file. Also, since netCDF-4 creates perfectly normal-looking
HDF5 files, any HDF5 visualization package can be used on a netCDF-4
file to look at the data. (I don't know if any visualization package can
display a data from a compound within a compound type!)
Yes, H5DUMP displays the directly correctly.
-Jeff
I will send out an update when this cross-platform work is complete.
Thanks!
Ed
--
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@xxxxxxxx
325 Broadway Office : Skaggs Research Cntr 1D-113
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg
