Work in maintaining, supporting, and developing the netCDF data model and software is associated with Endeavor 6: Improved scientific data access infrastructure from the Unidata 2008 proposal. NetCDF has become a key infrastructure element for producers and consumers of atmospheric science data, as well as data in other geosciences.
Recent netCDF development, both at Unidata and at other institutions, aims at improving interoperability with other representations for scientific data, making the netCDF interface more suitable for use on high-end parallel platforms with high-resolution models, and providing netCDF software on a wider range of desktop platforms.
We generated 4 beta releases for netCDF version 3.6.0, which included:
Documentation: In order to provide user and reference documentation for netCDF, we converted and updated the existing FrameMaker manuals to texinfo format and refactored the manuals into a language neutral guide and four language-specific reference manuals for C, Fortran-77 Fortran-90, and C++. From the texinfo source we can automatically generate HTML, PDF, and info versions.
NCSA and Unidata developers held several teleconferences and email discussions on proposed designs for HDF5 enhancements, HDF5 dimension scales, netCDF requirements for shared dimensions, support for coordinate systems, scale-offset compression, and packed data using HDF5 bit fields.
We prepared and presented a status update to the 2004 NASA Earth Science Technology Conference in Palo Alto in June. After the conference, we presented an annual report to NASA and an independent reviewer, a budget summary, a written annual report, and subsequent quarterly progress report.
Also in June, we made a description of tentative plans and current work on netCDF-4 is available on the netCDF-4 web site.
A tentative netCDF release schedule can be found at http://www.unidata.ucar.edu/software/netcdf/release_schedule.html.
Work continued on the development of a new version (2.2) of the Netcdf-Java library that uses an extended data model called the "Common Data Model", a generalization of the NetCDF, OPeNDAP and HDF-5 data models. Early versions of the code and documentation are available.
In September, a new project proposed by Charlie Zender (UC Irvine, PI) and Phil Papadopoulos (UC San Diego, Co-PI), "SEI(GEO): Scientific Data Operators Optimized for Distributed Interactive and Batch Analysis of Tera-Scale Geophysical Data" was funded by NSF's highly competitive Science and Engineering Informatics program. The goal is to develop a suite of Scientific Data Operators (SDO) for interactive and scripted manipulation of (locally and) widely distributed repositories of netCDF- and HDF5-formatted geophysical data. The "grid-enabled" SDO will accelerate distributed, network-transparent, analysis of ensemble tera-scale datasets stored at local and remote locations.
We continue to work with researchers at Argonne National Laboratory and Northwestern University who have developed a parallel interface for writing and reading netCDF data, tailored for use on high performance platforms with parallel I/O.
We hosted five visitors from Japan who were primarily interested in netCDF-4, but also in the IDV, LDM6, OPeNDAP, and THREDDS.