Work in maintaining, supporting, and developing the netCDF data model and software is associated with Endeavor 6: Improved scientific data access infrastructure from the Unidata 2008 proposal. NetCDF has become a key infrastructure element for data providers and users of oceanographic and atmospheric science data, as well as data in other geosciences.
Recent netCDF development, both at Unidata and at other institutions, aims at generalizing the netCDF data model, improving interoperability with other representations for scientific data, making the netCDF interface more suitable for use on high-end parallel platforms with high-resolution models, and providing netCDF software on a wider range of platforms.
The NetCDF CF Library supports the creation of scientific data files conforming to the CF conventions, using the netCDF API. It assists the user in correctly writing and interpreting the Climate and Forecast Conventions.
We designed and developed a libcf prototype library to help write and read CF conventions files. With the recent alpha4 release, the base functionality for the CF Convention document was achieved - the entire document has been addressed in some way (not necessarily the best, or final, way).
This is part of the general effort to bring some of the advanced features of the netCDF Java API to the C/Fortran netCDF library. The CF Library will allow users to use geo-located data in a more general way, without having to customize their tools (as much) for each data set.
NetCDF-4 is a major project to allow users to write and read data in HDF5 format, through the netCDF API. NetCDF-4 also expands the netCDF API to include new constructs, such as groups and user-defined data types. NetCDF-4 development has continued, with many alpha releases of netCDF-4.0 improving support for new platforms and compilers, performance, functionality, and documentation, and tracking changes in the HDF5 1.8.0 alpha releases.
With version 1.8-beta-1 of HDF5 just released (April 20, 2007), we are releasing the first beta test release of netCDF-4 under the assumption that the HDF5 file format on which netCDF-4 depends will remain fixed through the actual HDF5 1.8 release.
Two new training workshops, "NetCDF for Developers" and "NetCDF Java" were created and presented twice to a total of about 36 attendees. Feedback from the workshop participants was very positive. We're planning on a new form of workshop on July 20, for which participants will download and install software on their laptops before arriving. This will allow us to have a larger number of participants than is possible with rented workstations.
In March, 2007, we released netCDF 3.6.2, which prepares the netCDF configure and build system for the upcoming 4.0 release, and fixes some platform-specific build problems. It also includes complete example programs demonstrating the C, C++, Fortran-77, and Fortran-90 interfaces in a new netCDF tutorial for developers. Additionally, netCDF-3.6.2 includes support for shared libraries, support for multiple Fortran compilers on the same platform, C++ interface improvements, and better support for building Windows DLLs.
The nightly testing of the netCDF-3 and netCDF-4 packages at Unidata has been substantially improved. All available platforms now test netCDF with all available compilers, and with many variations of configuration options. The testing generates a daily snapshot of the netCDF distribution, and a daily report on the results.
For new users, we made new resources for learning about netCDF available: an online version of the NetCDF for Developers workshop, and a set of progressively more complex example programs available for various programming interfaces, initially C, Fortran-77, Fortran-90, C++, and Java. When we announced these, we also sought equivalent examples for other contributed interfaces, and we have made available the resulting examples for Python, MATLAB, Perl, and IDL.
With all the IPCC fourth assessment (AR4) data being archived and made available as CF-compliant netCDF files at Lawrence Livermore, Ray Pierrehumbert (University of Chicago) had this to say on realclimate.org:
... I think one mustn't discount a breakthrough of a technological sort in AR4 though: The number of model runs exploring more of scenario and parameter space is vastly increased, and more importantly, it is available in a coherent archive to the full research community for the first time. The amount of good science that will be done with this archive in the next several years is likely to have a significant impact on our understanding of climate.