The network Common Data Form (netCDF) was created in the late 1980s as a set of software libraries and self-describing, machine-independent data formats to support the creation, access, and sharing of array-oriented scientific data. Since that time, netCDF has been widely adopted in a variety of geoscience and other domains. In the early 2000s, the netCDF data model was extended to incorporate features of the OPeNDAP and HDF5 data models including Groups, user defined types, the String type, and unsigned integers. As a consequence of this change, the resulting netCDF version 4 (netCDF4) software chose to use the HDF5 file format to store data using this new model. Unidata and The HDF Group (maintainers of the HDF5 software) have a long and productive history of cooperation and joint activities in support of the geoscience community, of which this collaboration is only one instance.
In May 2018, The HDF Group announced a new support strategy for HDF5. The HDF5 libraries had previously been available as open-source software; this will continue to be true, and the fully open-source version of HDF5 will be referred to as the Community Edition (CE). In the future, however, The HDF Group will also provide a subscription-based Enterprise Support Edition (ESE) of HDF5. Subscribers to the ESE will receive feature updates to the core HDF5 modules before users of the CE, and additional HDF5 modules that add functionality outside the core of HDF5 may only be available to ESE subscribers. (Security and other important updates may be available to CE and ESE users at the same time.) Because HDF5 libraries are needed by the netCDF4 libraries to create fully-featured netCDF files, the changes to The HDF group's support strategy have raised questions about netCDF's future path in the netCDF community.
Unidata and the netCDF team have been in close contact with The HDF Group since their announcement, and we can state the following unequivocally:
- We reaffirm our commitment to open-source, freely-available software. Our goal is to create the lowest bar possible for using netCDF (and all Unidata software); to this end we minimize external dependencies when we can. We want to assure our community that we will avoid any software dependencies that require a paid license; Unidata will never publish a netCDF library that requires a license for paid software in order to create or read files that conform to the netCDF standard.
- Our commitment to 100% backwards read compatibility remains in effect. Any file conforming to the netCDF defined file formats and created by the netCDF library, or software using the netCDF library, will be readable by similarly-configured netCDF software in the future.
While Unidata and The HDF Group have chosen different paths in our software support and licensing policies, we respect their commitment to serving the geoscience community. Unidata will continue to work with The HDF Group to provide open-source scientific software that is both free to use and 100% backwards compatible with existing netCDF4 datasets. As part of this commitment, Unidata is pleased to have accepted an invitation to sit on the newly-formed HDF5 Technical Advisory Board.
