NetCDF

Status Report: April 2013 - September 2013

Russ Rew, Ward Fisher, Dennis Heimbigner

Strategic Focus Areas

The netCDF group's activities support Unidata's strategic goals in the following ways:

  1. Enable widespread, efficient access to geoscience data
    by developing netCDF and related cyberinfrastructure solutions to facilitate local and remote access to scientific data.
  2. Develop and provide open-source tools for effective use of geoscience data
    by supporting the use of netCDF and related technologies for analyzing, integrating, and visualizing multidimensional geoscience data; enabling visualization and effective use of very large data sets; and accessing, managing, and sharing collections of heterogeneous data from diverse sources.
  3. Provide cyberinfrastructure leadership in data discovery, access, and use
    by developing useful data models, frameworks, and protocols for geoscience data; advancing geoscience data and metadata standards and conventions; and providing information and guidance on emerging cyberinfrastructure trends and technologies.
  4. Build, support, and advocate for the diverse geoscience community
    by providing expertise in implementing effective data management, conducting annual training workshops, responding to support questions, maintaining comprehensive documentation, maintaining example programs and files, and keeping online FAQs, best practices, and web site up to date; fostering interactions between community members; and presenting community perspectives at scientific meetings, conferences, and other venues.

Activities Since the Last Status Report

Project and Issue Tracking

We use a project tracker tool to manage bug reports, track issues, plan releases, and make our development process more transparent to users. Between 13 April 2013 and 24 September 2013, we created 14 new issues, updated 45 issues, resolved 16 issues, and we currently have 73 open issues. (Note: issues vary greatly in size and effort required to resolve, so number of issues is not a useful measure of amount of work to do.)

We've migrated away from the previous home-grown testing scripts and web page for results of nightly testing to a new testing system based on CTest and a new CDash-based dashboard that provides more extensive reports on test results than the old system and will (eventually) allow user-submitted testing.

Workshops

We cancelled a "NetCDF for Developers" training workshop scheduled for July 23, after only 3 people registered for the course. We suspect lean travel budgets, vacations, and inclusion of "Developers" in the workshop title. We invited the 3 who had signed up to an informal meeting instead, to talk about netCDF plans and needs of the user communities they represented. The meeting was held on the morning of July 23, the day before the TDS-Python workshop.

Instead of a NetCDF Workshop, Russ and Ward presented sessions on accessing netCDF data with Python and iPython notebook server issues in the fully-subscribed TDS-Python workshop on July 24, along with 6 other UPC staff.

Releases

After the netCDF-C version 4.3.0 release in April, we moved the netCDF-C, Fortran, and C++ source repositories to GitHub, to improve collaboration with the user community. Since then, we have deployed and announced 3 release candidates for version 4.3.1, including various bug fixes, as well as portability, performance, and documentation enhancements, as described in the latest Release Notes. Evidence for improved collaboration includes several git "pull requests" from community developers contributing fixes.

Conference Presentations, Blogs, and Publications

In April, Ward presented a poster at EGU on "Information Visualization Techniques for Effective Cross-Discipline Communication". In August, Ward participated in the Front Range Consortium for Research Computing symposium in August. During the summer, Russ wrote three netCDF-related postings for the Unidata Developers' Blog.


Planned Activities

With Russ going to half-time on 1 October, the C-based netCDF project (which includes Fortran and C++ libraries, as well as netCDF utilities) will begin working with reduced resources, from 2.5 FTE to 2.0 FTE.

Ongoing Activities

  • Respond to support questions and help requests from netCDF users.
  • Improve support for netCDF on various platforms.
  • Incorporate successful features of netCDF-Java into C-based libraries.
  • Respond to needs of a growing user community for representing observational data, satellite products, and geoinformatics data.

New Activities

During the next six months, we plan to continue efforts to

  • finish CMake support on Linux for netCDF-Fortran
  • expand Windows support and implement Doxygen-generated documentation for netCDF-Fortran
  • improve Python example programs for netCDF-4
  • decide whether and how to continue maintenance of netCDF-4 C++
  • work on backlog of other unresolved issues entered into Jira for netCDF-C and netCDF-Fortran

Metrics

During the last 8 months, there were 65,300 downloads from 134 countries of the C-based netCDF software from Unidata, in addition to downloads from mirror sites, package management systems, and incorporation into other software packages. Detailed metrics, including for netCDF-Java/CDM, are available.

Other metrics, with comparisons from 6 months ago, include number of

  • Google hits for "netcdf": 936,000, down from 1,270,000
  • Google hits for "netcdf-3": 638,000, down from 763,000
  • Google hits for "netcdf-4": 605,000, down from 715,000
  • Bing hits for "netcdf": 271,000, up from 231,000
  • Blog mentions of netCDF: 22,800, up from 19,600
  • Google scholar entries for "netcdf": 9,960, up from 9,320
  • Books containing the term "netcdf": 8,330, up from 6,620
  • Free software packages that can access netCDF data: 82, down from from 84
  • Commercial software packages that can access netCDF data: 23, up from 21
  • U.S. patents mentioning netCDF: 48 (distinct), from 51 (included duplicates)
  • Coverity estimate, defects per thousand lines of code: 1.18, down from 1.47