Netcdf support et al,
Recent versions of netcdf-C are configured to write netcdf-4 files with
superblock version 0 and the earliest possible HDF5 internal object
formats. This was done for backward reader compatibility, in other words
to avoid superblock 3 and latest possible internal objects from HDF5
version 1.10.x, which could not be read by older software versions.
Before this change about a year ago, all netcdf-4 files were written with
superblock 2 and "latest" internal object versions from the HDF5 1.8 series.
I would like to propose a simple change in netcdf-C. Select the format
based on which HDF5 library version is linked to the netcdf-C library. In
shorthand:
If HDF5 library = 1.8.x
then default to H5F_LIBVER_LATEST; write files with superblock 2, etc.
else set H5F_LIBVER_EARLIEST; write files with superblock 0, etc.
With this change, software using the older HDF5 1.8 series library will
write netcdf-4 files that exactly match the internal format made by
previous netcdf-C and HDF5 library combinations. Superblock 0 files will
be created only when the new HDF5 1.10 library is used. This in a sense
gives users some control over which internal format to write, while keeping
up with the latest netcdf-C library improvements.
This proposal would enable a good format match with large archives of
netcdf-4 files that my agency previously published using the HDF5 1.8
series. Also, I am suspicious that files created with H5F_LIBVER_EARLIEST may
have reduced read back performance in some cases. I do not have current
examples or enough information about effects of the different internal
object versions to be able to assess this properly. However, performance
issues were hinted in a recent HDF5 discussion.
The HDF Group said they will try to implement more fine grained format
control in some future HDF5 release, but there is no schedule for that.
If anyone sees a problem with this interim proposal, please let us know.
Otherwise, please consider adding this format control to netcdf-C for the
upcoming 4.5 release. Thanks for your consideration.
--Dave
