NetCDF

Status Report: April - September 2010

Russ Rew, Ed Hartnett, Dennis Heimbigner

Strategic Focus Areas

The netCDF group's work supports the following Unidata funding proposal focus areas:

  1. Broadening participation and expanding community services
    by providing effective support to a growing netCDF user community, including users of satellite products, GIS, climate, and ocean data.
  2. Advancing data services
    by developing and supporting a key infrastructure element for data providers and users of climate, atmosphere, and ocean data.
  3. Developing and deploying useful tools
    by engineering general-purpose software and utilities supporting the analysis, visualization, and management of model output and observational data.
  4. Enhancing user support services
    by maintaining comprehensive documentation, providing annual workshops, responding to support questions, maintaining example programs and files, and keeping online FAQs, best practices, and the netCDF web site up to date.
  5. Providing leadership in cyberinfrastructure
    by seeking endorsement of standards bodies for netCDF formats and for Climate and Forecast (CF) model metadata conventions based on netCDF.
  6. Promoting diversity by expanding opportunities
    using an open source development process, accepting user contributions, and democratizing access to valuable data archives.

Activities Since the Last Status Report

Two releases

The major feature in the 4.1.1 release was support for remote data access with the new built-in OPeNDAP client. This is important, because:

  • it is more portable and easier to install than a C++-based alternative from opendap.org, which it replaces; and
  • it provides users of C-based netCDF software access to Java-based server technologies such as NcML aggregation of multiple files into a single virtual file, access to data in other formats such as GRIB and BUFR through a netCDF interface, and the netCDF subset service.

The 4.1.1 also release included many other new features.

The 4.1.2-beta release contained documentation improvements, bug fixes, performance improvements, and a major internal refactoring (see below).


Refactored library dispatch architecture

Without changing the programming interfaces, this improved the architecture of the C library for planned extensions (such as reading other formats through netCDF interfaces), reduced the size of the code by eliminating duplication, and improved maintainability.


Moved to svn

The C-based development code was made publicly available from an svn server, to improve transparency and provide better support for open source development.


libCF/GRIDSPEC support in netCDF

Unidata continued overseeing development of the GRIDSPEC "Princeton API" in libCF as part of a collaboration with NOAA's Global Interoperability Project, PCMDI, GFDL, and TechX, anticipating that this will become part of the CF conventions. This included:

  • Established collaborative development environment with TechX developers, including SourceForge code repository
  • Worked with TechX programmers to integrate GRIDSPEC Princeton API functions into libCF
  • Submitted abstract for an AGU poster on GRIDSPEC/libCF

Presentations

In May, Russ presented Adapting Software to NetCDF's Enhanced Data Model at the annual EGU meeting. In September Ed presented HDF5 and NetCDF-4 at the HDF5 annual workshop.

Planned Activities

Ongoing Activities

We plan to continue the following activities:

  • Respond to C- and Fortran-based netCDF user questions and run two netCDF workshops.
  • Incorporate successful features of netCDF-Java into C-based libraries.
  • Improve support for evolving Climate and Forecast (CF) conventions.
  • Foster standardization of netCDF and CF conventions through the Open Geospatial Consortium, NASA Standards Process Group, and other relevant standards bodies.
  • Provide support for writing and reading GRIDSPEC multifile datasets, as mandated for climate modeling initiatives.
  • Deal with needs of a growing user community for representing observational data, satellite products, and geoinformatics data.

New Activities

We plan to undertake the following:

  • Port C-based netCDF-4 to Windows.
  • Create new netCDF-4 test file collection to support netCDF-4 tool developers.
  • Refactor netCDF Fortran-90 API to use Fortran-2003 C-interoperability layer.
  • Integrate contributed netCDF-4 C++ API into distribution
  • Convert documentation from texinfo to Doxygen.

Relevant Metrics

During the last year, there were about 93,000 downloads from 114 countries of the C-based netCDF software from Unidata, in addition to downloads from mirror sites, package management systems, and incorporation into other software packages. More detailed metrics are available.

Other metrics that may be useful include:

  • Number of open-source software packages that can use netCDF data: 76
  • Number of commercial or licensed software packages that can use netCDF data: 21
  • Number of Google hits for "netcdf": 3,000,000
  • Number of Google images for "netcdf": 9,110
  • Number of Google scholar entries for "netcdf": 6,090
  • Google count of number of books containing the term "netcdf": 2910
  • Amazon count of number of books relevant to the subject "netcdf": 18
  • Number of Colorado license plates containing "NETCDF": 1