« AWIPS II 14.2.1 Beta... | Main | Unidata Welcomes New... »

Version 4.4.5 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.

The NCO project is coordinated by Professor Charlie Zender of the Department of Earth System Science, University of California, Irvine. More information about the project, along with binary and source downloads, are available on the SourceForge project page.

From the release message:

This release is mainly a bugfix and stability release. A few new features improve some corner-cases: NCO now supports longer lists of input files, DAP files on HTTPS servers, and the CF ancillary_variables convention.

New Features

1. NCO now supports stdin input file lists up to ~50 MB long. This applies to all NCO multi-file operators (ncra, ncea, ncrcat). The previous limit was 10 MB. The new limit suffices for 30 years of hourly datasets where each dataset name is 192 bytes long. One NASA user is averaging 10 years of data, or 87600 files. Who will be the first user to break the million file barrier?
   http://nco.sf.net/nco.html#stdin
2. NCO now supports the CF ancillary_variables convention described at http://cfconventions.org/1.6.html#ancillary-data This convention allows ancillary variables to be associated with one or more primary variables. NCO attaches any such variables to the extraction list along with the primary variable and its coordinates and bounds, if any.
   http://nco.sf.net/nco.html#ncl
3. NCO now accesses DAP files on HTTPS servers. Previously NCO would only test filenames beginning with "http://" for DAP access. Now NCO also tests filenames beginning with "https://" for DAP. If you use DAP servers with a protocol besides HTTP or HTTPS please tell us so we can enable NCO for those services too.
   http://nco.sf.net/nco.html#dap
4. ncks --grp_xtr_var_xcl: Group Extraction Variable Exclusion. This new switch helps control variable and group extraction. Normally NCO works in "intersection mode" where the objects extracted are the intersection of arguments to -g and -v. And the --union switch can be used to make their union. As of 4.4.4 (June, 2014), ncks (alone) supports an option to include specified groups yet exclude specified variables. The --grp_xtr_var_xcl switch (with long option equivalent --gxvx extracts all contents of groups given as arguments to -g except for variables given as arguments to -v. Use this when one or a few variables in hierarchical files are not to be extracted, and all other variables are. This is useful when coercing netCDF4 files into netCDF3 files such as with converting, flattening, or dismembering files.
   ncks --grp_xtr_var_xcl -g g1 -v v1 # Extract all of group g1 except v1
   http://nco.sf.net/nco.html#gxvx
5. New chunking policy r1d changes chunksize of 1-D record variables (and no other variables) to specified (with --cnk_dmn) chunksize. Any specified record dimension chunksizes will be applied to 1-D record variables only. Other dimensions may be chunked with their own --cnk_dmn options that will apply to all variables. For example, ncks --cnk_plc=r1d --cnk_dmn=time,1000. This sets time chunks to 1000 only in 1-D record variables. Without the r1d policy, time chunks would change in all variables. Not to be confused with chunking map rd1. Thanks to James Adams for suggesting this.
   http://nco.sf.net/nco.html#r1d

Bug Fixes

1. ncrename: fix unsafe pointer de-reference and improve diagnostics
2. Fix problem introduced in 4.4.0 where sanitizing of filenames with colons confused MS Windows interpretation of volume names, e.g., "C:\foo" became "C\:\foo" and caused write error. Workaround is to use --no_tmp_fl or upgrade to 4.4.5. Thanks to James Adams for reporting.
3. Always send correct dimension sizes to chunker when hyperslabbing. Fixes a bug reported where chunking and hyperslabbing played cruelly together. Thanks to Parker Norton for reporting.
   http://nco.sf.net#bug_cnk_hyp
4. ncpdq: fix mishandling of dimension reversal arguments Versions 4.3.2-4.4.4 of ncpdq could fail to correctly parse some valid forms of the dimension reversal syntax. Failure resulted, so no bad data was written. Solution is to upgrade to 4.4.5. Thanks to Fan Fang for reporting.

Additional details are available in the ChangeLog.