« AWIPS Tips: Using... | Main | IDV Users: Please... »

Version 5.0.3 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.

The NCO project is coordinated by Professor Charlie Zender of the Department of Earth System Science, University of California, Irvine. More information about the project, along with binary and source downloads, are available on the SourceForge project page.

From the release message:

Version 5.0.3 contains significant improvements to the quantization algorithms and fixes to MOAB/mbtempest regridding. NCO now supports the Digit Round and Granular BitGroom quantization algorithms. The failed self-test problems that prevented Conda from building NCO 5.0.2 have been fixed, and a 5.0.3 Conda package will soon be available.

New Features

1. Lossy quantization algorithms have been refactored. First, the attribute name that store the level of quantization are now algorithm-specific, e.g., QuantizeBitGroomNumberOfSignificantDigits. Previously, NCO stored the NSD attribute number_of_significant_digits. However, this was deemed too ambiguous, given the increasing number of supported quantization methods. The new attribute names better disambiguate which algorithm was used to quantize the variable. Second, the default algorithm has been changed from BitGroom with BitRound masks from R. Kouznetsov (2021), to what we call Granular BitGroom (GBG). GBG combines features of BitGroom, BitRound, and DigitRound by Delaunay et al. (2019). GBG improves compression ratios by ~20% relative to BitGroom for NSD=3 on our benchmark 1 GB climate model output dataset.

   ncks -7 -L 1         --ppc default=3 in.nc out.nc # GranularBitGroom
   ncks -7 -L 1 --baa=0 --ppc default=3 in.nc out.nc # BitGroom
   ncks -7 -L 1 --baa=1 --ppc default=3 in.nc out.nc # BitShave
   ncks -7 -L 1 --baa=2 --ppc default=3 in.nc out.nc # BitSet
   ncks -7 -L 1 --baa=3 --ppc default=3 in.nc out.nc # DigitRound
   ncks -7 -L 1 --baa=4 --ppc default=3 in.nc out.nc # GranularBitGroom
   ncks -7 -L 1 --baa=5 --ppc default=3 in.nc out.nc # BitRound
   ncks -7 -L 1 --baa=6 --ppc default=3 in.nc out.nc # HalfShave
   ncks -7 -L 1 --baa=7 --ppc default=3 in.nc out.nc # BruteForce
   

   Documentation on Bit-Adjustment-Algorithms (BAA) 3-7 will be improved in the future. Bit-Groom/Shave/Set are described here:
   http://nco.sf.net/nco.html#ppq
   http://nco.sf.net/nco.html#gbg

Additional details are available in the ChangeLog.