« MetPy 1.2.0 Released | Main | AWIPS Tips: Access... »

25 January 2022

Version 5.0.5 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.

The NCO project is coordinated by Professor Charlie Zender of the Department of Earth System Science, University of California, Irvine. More information about the project, along with binary and source downloads, are available on the SourceForge project page.

From the release message:

Version 5.0.5 brings a new lossy quantization codec, BitRound. Granular BitGroom has been renamed Granular BitRound. ncclimo has improved API consistency between climo generation and timeseries extraction. ncatted introduces a new editing mode: prepending. If these issues do not interest you, you might skip 5.0.5.

New Features

Arithmetic operators now support the lossy quantization algorithm known as BitRound. The BitRound algorithm is described in "A note on precision-preserving compression of scientific data" by R. Kouznetsov, doi:10.5194/gmd-14-377-2021. BitRound has been used to quantize more accurately in NCO's BitGroom and GranularBR algorithms for over a year. However, though algorithms select the number of bits to quantize internally. Now BitRound is an independently configurable algorithm in its own right. It takes a user-specified number of significant bits (NSB, or "keepbits) to retain. The NSB argument contrasts with the number of significant digits (NSD) parameter expected by BitGroom, DigitRound, and GranularBR. BitRound is Bit-Adjustment-Algorithm #8.
```
ncks --baa=8 --ppc default=9 in.nc out.nc # BitRound NSB=9
ncks --baa=4 --ppc default=3 in.nc out.nc # GranularBR NSD=3
```
Thanks to Milan Klower and R. Kouznetsov for prototyping BitRound in the first place.
ncclimo now accepts filenames from stdin or positional arguments in monthly climo mode. Previously, ncclimo self-generated input filenames for monthly climos, given the caseid, and start/end years. Now ncclimo can be fed a list of filenames to climatologize. The API is identical to providing filenames to the splitter: pipe filenames through stdin or place them on the command-line. This makes the climo generator even more flexible than the splitter which cannot (yet) generate filenames internally. However, the flexibility of providing filenames via stdin or positional arguments is more error-prone than using internally generated filenames that are, by construction, guaranteed to be in the correct sequence. User-provided filenames are vetted en masse to ensure that the expected number of files for the specified climatology are present. They are not vetted to ensure that they are in the correct order, i.e., January then February then March of the correct year etc. The $caseid argument is optional when the user provides the files. However, ncclimo needs to know what to name the output files. The $caseid argument or the $fml_nm argument, when present, provides the output filename stub. When neither of these options are present ncclimo will name the output files after the model name, if known.
```
ls mpaso*.nc | ncclimo -P mpaso -s 50 -e 100 -o out # Stdin
ls eam*h0*.nc | ncclimo --fml=name -s 50 -e 100 -o out # Stdin
ncclimo -P mpaso -s 50 -e 100 -o out mpaso*.nc # Positional
ncclimo --fml=name -s 50 -e 100 -o out eam*h0*.nc # Positional
```
http://nco.sf.net/nco.html#ncclimo
ncatted has a new edit mode: Prepend. Analogous to append mode, prepend mode will prepend the user-specified value to the attribute and if the attribute does not exist it will be created:
```
ncatted -a char_att,att_var,p,c,"this is prepended.\n" in.nc out.nc
```
http://nco.sf.net/nco.html#att_prepend