NetCDF-4 Performance Improvements

13.9 NetCDF-4 Performance Improvements

NetCDF-4 uses HDF5 for its storage layer, which provides several ways to improve performance, even for netCDF-3 programs.

Per-variable compression
Per-variable chunking (multidimensional tiling)
Parallel-I/O (HDF5's, not Argonne's Parallel netCDF) for platforms with parallel file systems
"Reader makes right" rules
Efficient dynamic schema changes
Performance benefits for writers creating "classic model" files
Performance benefits for netCDF-3 reader programs reading "classic model" netCDF-4 (HDF5) files by relinking with the netCDF-4 library

Early users of netCDF-4 have reported 4:1 compression and 7x speedups for model data storage and access.