Hi Joe,
> NCAR/EOL/CDS is interested in using the compression features of
> NetCDF-4 with the Classic NetCDF Data Model so our users could
> potentially use third party tools to process our data. (Using
> compression requires the data to be stored in netCDF-4/HDF5 format,
> even if the Classic NetCDF Data Model is used.)
>
> 1) Is it true that any program written with the netCDF3 API needs
> minimal (no?) changes to read such a netCDF-4 compressed file, once
> the application has been relinked with the netCDF-4 library?
Yes, that's true. Any netCDF-4 file created with the flag that marks it
as conforming to the classic model can be read (or written) by a
netCDF-3 program that has been relinked to the netCDF-4 library. Such a
file can use any of the so-called "performance features" of netCDF-4,
but none of the enhanced data model features.
The performance features that classic model files can use include
chunking (multidimensional tiling), efficient per-variable compression,
efficient schema changes, and ample variable sizes. Enhanced netCDF-4
data model features that such classic model files are *prevented* from
using include multiple unlimited dimensions, unsigned integers and
64-bit integers, groups for nested scopes, compound types (structs),
variable-length types (supporting ragged arrays), or enumeration types
(for compact self-description).
> 2) Can anyone tell me if programming languages like IDL or MATLAB have
> been linked with the netCDF-4 library? (I suspect not, but I haven't
> been able to determine this by running 'nm' or 'ldd').
>
> 3) Has anyone heard discussions about IDL supporting netCDF-4 or using
> HDF5 1.8.1?
We've just learned that ITT Visual Information Solutions, the company
that develops and maintains IDL, has an open feature request (cr43022)
to support netCDF-4. They will look more closely at the request after
the IDL 7.1 release this summer. We think a good strategy would be to
provide netCDF-4 support in two stages, first taking the easy step of
just relinking against the netCDF-4 library. That would support classic
model file access and satisfy a lot of user needs, in particular
efficient access to compressed data. However, it may be more
complicated than that, because IDL's HDF5 support may have to be
upgraded to the new 1.8 APIs at the same time.
> 4) Has anyone heard discussions about MATLAB supporting netCDF-4 or
> using HDF5 1.8.1?
We would also be interested in any information about updates to MATLAB
plug-ins to support netCDF-4 or MathWorks plans for netCDF-4 support,
especially the classic model support that only requires relinking.
> 5) What's the "market penetration" of netCDF-4? Does anyone have a
> feel for how many netCDF-3 users have at least relinked with the
> netCDF-4/HDF5 libraries, even if they aren't using the more advanced
> netCDF-4 features, like groups?
It looks as if netCDF-4 will be used for GOES-R ground-segment data
products, and the netCDF-4 classic model format has recently been
endorsed for use by NOAA NESDIS as the standard intermediate file format
for new satellite products from the Center for Satellite Applications
and Research operational systems and from the NOAA Office of Satellite
Data Processing and Distribution. Use of the netCDF-4 classic model is
also under consideration by several ocean and climate modeling projects.
--Russ
