Version 5.3.1 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.
The NCO project is coordinated by Professor Charlie Zender of the Department of Earth System Science, University of California, Irvine. More information about the project, along with binary and source downloads, are available on the SourceForge project page.
From the release message:
Version 5.3.1 adds features and fixes to ncremap and ncclimo.
The new autoconversion feature allows these scripts to automagically
convert netCDF4 string variables to netCDF3 character arrays.
The improved treatment of fixed time dimensions allows ncclimo to work
with raw MERRA2 files. Together these improvements make possible
regridding and vertical interpolation of the new formatting of
Copernicus Data Store (CDS) files used for, e.g., ERA5.
Skip this release if these changes do not interest you.
New Features
-
Autoconversion of NC_STRING variables, i.e., automatic conversion
from type NC_STRING to NC_CHAR as dictated by choosing a netCDF3
output from a netCDF4 input file, work for all attributes, but not for
all variables. This is because attributes are at most one-dimensional
and may be of any size whereas variables require gridded dimensions
that usually do not fit the ragged sizes of text strings.
Hence scalar NC_STRING attributes are correctly converted to and
stored as NC_CHAR attributes in the netCDF3 output file, but
not all NC_STRING variables are correctly converted.
As of version 5.3.1, NCO has two distinct levels of support for autoconversion of NC_STRING variables.ncremapfully supports autoconversion of scalar NC_STRING variables in both horizontal regridder and vertical interpolation mode. This means thatncremapwill produce netCDF3 output files from netCDF4 input files (if requested) and that any NC_STRING variables in the input files will be translated to NC_CHAR arrays in the netCDF3 output files. There is no loss of information since the entire contents of the string is preserved in the character array. Autoconversion stores the character array with a new dimension namedsng_lng_Xwhere X is the smallest power of ten between 10 and 10000 that is capable of containing the string contents (including the terminating NUL character). This type of variable autoconversion fails if the input string length exceeds 10000 (this is a safety measure, and could be expanded if people lobby me).
All operators besidesncremapcurrently retain only the first character of the input NC_STRING array. This loses information. We are working to implement the same feature thatncremapenjoys. Both types of autoconversion fail if the input string variable is multidimensional, i.e., an array of type NC_STRING. A feature to solve that case is tractable, though we have not yet received any requests for it.
http://nco.sf.net/nco.html#autoconversion
http://nco.sf.net/nco.html#ncclimo -
ncremap's vertical interpolator now automatically extrapolates variables named "t" (as in ERA5 raw data) as if they were temperature. Futhermore the interpolator now automatically extrapolates variables named "H" (as in MERRA2 raw data) and VerticalLayerMidpoint (SCREAM) as if they were geopotential height. See the documentation for specifics about the temperature and geopotential height extrapolation algorithms.
http://nco.sf.net/nco.html#vrt -
For many years, users at select supercomputer centers have been
able to access the latest versions of NCO scripts directly from
C. Zender's build directories. However, this feature was not
documented until now. Zender's build directories usually contain the
latest NCO snapshot, and so may be unstable and are not for everyday
use. This feature is instead intended to allow users to test and
provide feedback on new features before they are distributed in a
public release.
Invokingncclimoandncremapwith the--npo(or long-option equivalent--nco_path_override) flag as the first option (i.e., as $1) causes these scripts to utilize the binary NCO executables in Zender's build directories on the following machines at the national supercomputer centers in parentheses:
acme1 (LLNL), andes (ORNL), chrysalis (ANL), compy (PNNL), derecho (NCAR), frontier (ORNL), and perlmutter (NERSC).
If the latest features that you want are actually implemented in the NCO binaries or library, then it may suffice to invoke your own version of the scripts, as in the first examples below. However, if the latest features that you want are implemented in the scripts themselves, then you can either invoke Zender's scripts directly (as in the second set of examples), or copy those scripts into your own executable path:# Access Zender's latest binaries from your default scripts ncremap --npo -m map.nc in.nc out.nc ncclimo --npo -P elm -c ${caseid} ... # Access Zender's latest binaries from Zender's latest scripts ~zender/bin/ncremap --npo -m map.nc in.nc out.nc ~zender/bin/ncclimo --npo -P elm -c ${caseid} ...In both cases, the--npoflag must be the first option invoked.
http://nco.sf.net/nco.html#npo
http://nco.sf.net/nco.html#nco_path_override
Additional details are available in the ChangeLog.
