netCDF Operators NCO version 5.1.0 are brachiating toward you
http://nco.sf.net (Homepage, Mailing lists, Help)
http://github.com/nco (Source Code, Issues, Releases, Developers)
What's new?
Version 5.1.0 supports the new netCDF 4.9.0 features for filters and
quantization. This is a significant new feature that justifies the
version bump from 5.0.7 to 5.1.0. There is also an ncremap fix for
sub-sub-gridscale regridding, e.g., landunit-specific regridding.
Users of sub-SGS regridding, and anyone interested is using modern
compression or quantization algorithms such as Bzip2, Zstandard,
Granular BitGroom, and BitRound.
Work on NCO 5.1.1 has commenced and will polish support for
quantization/compression algorithms and add support for NCZarr I/O.
Enjoy,
Charlie
NEW FEATURES (full details always in ChangeLog):
A. All numeric NCO operators support new compression features
when linked to a default installation of the latest netCDF, 4.9.0.
These features and how to access and tune them are extensively
documented in the manual. Suffice it to say that all NCO users
now have transparent access to modern lossless (Bzip2, Zstandard)
and lossy (BitRound, Granular BitRound) compression algorithms.
Codecs from the netCDF C-library, the Community Codec Repository
(CCR), and from user-installed HDF5 filters may all be used.
The commands will look something like this:
ncks --cmp='granularbr,3|shuffle|zstandard,3' ... # Long-names
ncks --cmp='gbr,3|shf|zst,3' ... # Abbreviations and levels
ncks --cmp='gbr|shf|zst' ... # Default levels
ncks --cmp='f32|gbr|shf|bz2|dfl|zst' ... # Multiple codecs
ncwa --cmp='gbr|shf|zst' ... # Other operators...
ncbo --cmp='gbr|shf|zst' ... # All numeric operators, in fact
ncclimo --cmp='btr\|shf\|zstd' ... # Climo generator
ncremap --cmp='btr\|shf\|zstd' ... # Regridder
Full documentation (including figures) is at
http://nco.sf.net/nco.html#compression
Thanks to Ed Hartnett (NOAA) and Dennis Heimbigner (Unidata) for
guiding the addition of these features into the CCR and netCDF.
BUG FIXES:
A. NCO 5.0.7 contained a bug in its new feature that allows missing
values in fields with valid SGS area. ncremap < 5.0.7 assumed fields
to be valid wherever sgs_frc > 0.0. This assumption is valid for ~95%
of ELM/CLM fields. However, it low-biased landunit-specific fields
(like TSOI, H2OSOI) in transition areas (e.g., ablation zone) with
valid SGS, and invalid landunit. NCO 5.0.7 mostly fixed that, though
it had a corner-case bug triggered mostly in multi-level fields
(like TLAKE, TSOI). The normalization for multi-level fields is now
correct, thus allowing sub-sub-gridscale regridding to work in all
known cases.
ncremap -P elm --map=map.nc in.nc out.nc
http://nco.sf.net/nco.html#sgs
Full release statement at http://nco.sf.net/ANNOUNCE
--
Charlie Zender, Earth System Sci. & Computer Sci.
University of California, Irvine 949-891-2429 )'(
