Work in maintaining, supporting, and developing the netCDF data model and software is associated with Theme 2: Advancing data services from the Unidata 2013 proposal. NetCDF continues to serve as a key infrastructure element for data providers and users of climate, atmosphere, and ocean data.
Recent netCDF development continue to improve client access to remote data services, documentation, and interoperability with other representations for scientific data. Other work provides client access to remote data services through the netCDF interface, improves use on high-end parallel platforms with high-resolution models, and maintains compatibility of netCDF software on a wide range of platforms.
NetCDF-4, a major upgrade to netCDF, was released in June 2008. It uses the HDF5 format to read and write data files through the netCDF programming interfaces. This allows the introduction of many new features, including compression, parallel I/O, multiple unlimited dimensions, user-defined data types, and more. NetCDF-4 is a drop-in replacement for netCDF-3.x, because it is fully backward compatible.
Recent development work has improved performance, improved the way parallel I/O works, refined the Fortran interfaces, improved the build system, and enhanced the ncdump utility available to handle all the new features of the netCDF-4 data model.
A few bugs have been found and fixed in the netCDF snapshot release. These will be included in a formal version 4.0.1 release in October. We have also developed and published plans for netCDF releases 4.1, 4.2, 4.3, and 4.4.
In support of the CF conventions for climate calendars, we incorporated software from LLNL into ncdump to optionally display time units in human-readable form, such as "2008-10-21 06:00" instead of seconds or days since an epoch, in addition to supporting common climate modelling calendars.
User documentation has also been extensively revised. IN addition, a draft white paper "Developing Conventions for netCDF-4" discusses the issues in evolving metadata conventions to the enhanced netCDF data model.
We have recently made significant progress on an NSF-funded cyberinfrastructure project for improving OPeNDAP and netCDF integration, and an integrated DAP library is now included with the latest netCDF-4 snapshot releases (4.1-alpha). It has begun to accomplish the first goal of the project, to enhance Unidata's netCDF C library to support the OPeNDAP protocol as the remote access protocol for netCDF data.
Using this software, it is now possible for a C or Fortran program that reads netCDF data to access specified subsets of remote data in a variety of formats provided by OPeNDAP servers, including Unidata's THREDDS Data Server (implemented in Java).
This was possible previously, but required getting and building a separate OPeNDAP library first, enough of an obstacle to prevent many users from taking advantage of the OPeNDAP protocols.
Another goal of the project is to extend the OPeNDAP protocol to support the Unidata Common Data Model on the server side as well as the client side. Implementation of this extension is being accomplished by opendap.org.
Remaining Unidata work includes replacing the C++ implementation of the DAP C++ client code with more a more portable C library that will build on more platforms and work more easily with Fortran applications. We have recently modified the C library (Ocapi) to revise the parser, fix bugs, and plug memory leaks, and contributed the fixes back to opendap.org for incorporation into the latest Ocapi release.
We submitted a "Proposal for Development of Climate and Forecast Conventions for Satellite Data" to NASA ESTO for earth observing platforms mentioned in the 2007 decadal survey. We also wrote to the NOAA GOES-R ground segment project and the NOAA data management project for "remanifesting" sensors on the NASA decadal survey satellites, requesting potential collaboration and availability of resources for conventions and libcf development.
Version 2.2.22 of the netCDF-Java library was released in February 2008. Recent versions have improved BUFR support and refined other I/O Service Provider classes to complete implementation of the Common Data Model. Work continues on NetCDF-Java 4.0, with release expected in Fall 2008.
by Russ Rew