« Unidata Users Worksh... | Main | MetPy: An Open Sourc... »

Version 4.4.9 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.

The NCO project is coordinated by Professor Charlie Zender of the Department of Earth System Science, University of California, Irvine. More information about the project, along with binary and source downloads, are available on the SourceForge project page.

From the release message:

This release includes small feature improvements in chunking, deflation, and attribute handling during dimension reduction. Also improvements in clarity and consistency of messages. While chunking is an esoteric feature for many, its importance is increasing as the move to netCDF4 (classic) continues. The chunking defaults and flexibility have reached a new level of robustness in this release.

The main new awesome feature is that ncra understands weights. Finally. Applying uneven weights to different input files was too clunky. This release is dedicated to the intrepid souls who use NCO to generate climatologies that require uneven weights. This release should make that task much easier.

New Features

1. NCO has improved default performance of deflation and chunking. Our "best practices" chunking ( map/policy=nco ) is now default when manually deflating netCDF3 files into netCDF4 files. This invokes Rew's balanced chunking for three-dimensional variables, and LeFter-Product ( cnk_map=lfp ) chunking for others. LFP chunking, in turn, implements more reasonable defaults when the variable size is smaller than a single chunksize. This reduces excessive chunk sizes for many small variables. Shuffle is now turned-on by default when manually deflating files. This restores behavior from the NCO 4.3.x series, and can improve compression ratios by 10-30% relative to not shuffling. Copying multi-dimensional record variables from netCDF3 to netCDF4 files no longer invokes the MM3 workaround. This speeds-up deflation of netCDF3 datasets.
   
   ncks -O -4 -L 1 netCDF3.nc netCDF4.nc
   
   http://nco.sf.net/nco.html#cnk
2. ncwa now eliminates dimensions from the "coordinates" attribute after collapsing them. This helps ensure that rank-reduced variables become completely independent from their former dimensions. The former presence of collapsed dimensions continues to be indicated by the "cell_methods" attribute.
   
   ncwa -O -a lat in.nc out.nc
   
   http://nco.sf.net/nco.html#coordinates
   http://nco.sf.net/nco.html#cell_methods
   http://nco.sf.net/nco.html#ncwa
   
3. NCO now warns users and suggest workarounds in situations when operators behave correctly, though perhaps not as expected, including:
   1. When coordinate variables intended to be excluded by -x and -C may nonetheless appear in output file, and suggest workaround
   2. When hyperslabs are specified for multi-dimensional "coordinates"
4. NCO now respects the CF "climatology" attribute: The variable pointed to by this attribute is treated as a pseudo-coordinate variable, and is extracted by default with any referring variable. Climatology variables now obey the same arithmetic rules as coordinates and are exempted from certain operations. Their treatment is identical to that of "bounds" variables. This command now retrieves the named variable, its coordinates (if any), and any variables named in the "ancillary_variable", "bounds", "climatology", and "coordinates" attributes:
   
   ncks -v temperature in.nc out.nc
   
   http://nco.sf.net/nco.html#climatology
5. ncra now accepts user-specified weights with the -w switch. When no weight is specified, ncra continues its old behavior and weights each record (e.g., time slice) equally. Weights specified with -w wgt may take one of two forms. First, the wgt argument may be a comma-separated list of values by which to weight each input file. Or, the wgt argument may be the name of a weighting variable present in every input file. The variable may be a scalar or a one-dimensional record variable. Scalar weights are applied uniformly to the entire file (i.e., a per-file weight), while one-dimensional weights apply to each corresponding record (i.e., per-record weights). Two applications of weights with ncra include easier generation of accurate averages of seasonal statistics, and of accurate averages of quantities sampled with a dynamically changing timestep.
   
   ncra -w 31,31,28 dec.nc jan.nc feb.nc out.nc
ncra -w delta_tm in1.nc in2.nc in3.nc out.nc
   
   http://nco.sf.net/nco.html#ncra

Additional details are available in the ChangeLog.