« Research Assistant... | Main | NetCDF 4.4.0 »

Version 4.5.4 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.

The NCO project is coordinated by Professor Charlie Zender of the Department of Earth System Science, University of California, Irvine. More information about the project, along with binary and source downloads, are available on the SourceForge project page.

From the release message:

We are pleased to introduce ncremap , a new operator for regridding. ncremap is a shell script that wraps ncks to produce a friendly interface to regridding features. Without any external dependencies, ncremap applies weights from a pre-exisiting mapfile to a source data file to produce a regridded dataset. Source and destination datasets may be on any Curvilinear, Rectangular, or Unstructured Data (CRUD) grid. When necessary ncremap calls ESMF_RegridWeightGen or TempestRemap to generate weights and uses those to regrid. ncremap hides the complexity of regridding from the user, who can know next-to-nothing about regridding.

New Features

1. ncremap : A new netCDF operator for regridding.
   ncremap is a shell script that wraps ncks to produce a friendly interface to regridding features. Without any external dependencies, ncremap applies weights from a pre-exisiting mapfile to input data file(s) to produce a regrided dataset(s). Source and destination datasets may be on any Curvilinear, Rectangular, or Unstructured Data (CRUD) grid. ncremap will also, when necessary, use external programs (ESMF's ESMF_RegridWeightGen (ERWG), or TempestRemap's GenerateOverlapMesh/GenerateOfflineMap) to generate weights and mapfiles and use those to regrid. ERWG is distributed in binary format with NCL, which many (most?) users already have on their system. Or ERWG and TempestRemap may be installed from source. Please try ncremap and send us your feedback and suggestions.
   
   Examples:
   

   ncremap -i src.nc -d dst.nc -o out.nc
   ls gcm14*cam*0007*.nc | ncremap -a conserve -M -d dst_1x1.nc -O ~/rgr

   http://dust.ess.uci.edu/smn/pst_nco_agu_201512.pdf # AGU Poster
   http://nco.sf.net/nco.html#ncremap
2. ncks prints XML "_Unsigned" attribute for unsigned attribute types. Formerly, ncks did not preserve the signedness of attributes.

   % ncks -v att_var --xml ~/nco/data/in_4.nc | grep Unsigned
   <attribute name="ubyte_att" type="byte" isUnsigned="true" value="0 1 2 127 128 254 255 0" />
   <attribute name="ushort_att" type="short" isUnsigned="true" value="37" />
   <attribute name="uint64_att" type="long" isUnsigned="true" value="0" />

   http://nco.sf.net/nco.html#xml
3. ncap2 now accepts [] and () syntax interchangeably. Expressions which formerly had to use one or the other can now use either.
4. ncap2 now conforms arrays by hyperslabbed dimension size rather than by dimension name. This mean, e.g., that hyperslabs from one dimension can be used to fill-in other dimensions. A prototypical example is determining pressure thickness of layers as the difference between interface pressure levels. For hybrid coordinate system models like CAM, this is now a (relatively) simple two-step command with a mixture of different dimensions on the LHS and RHS:
   

   prs_ntf[time,lat,lon,ilev]=P0*hyai+PS*hybi; // Interface pressures
   
   prs_dlt[time,lat,lon,lev]=prs_ntf(:,:,:,1:$ilev.size-1)-prs_ntf(:,:,:,0:$ilev.size-2);

   http://nco.sf.net/nco.html#pdel
5. NCO now treats variables named wgt_* as weights and avoids performing math on them when possible. E.g., ncbo will preserve (and not difference) the variable wgt_1 . This is the same behavior that NCO uses for coordinates, "gw", and masks (named with msk_*.) The idea is that weights are a property of the grid and should not be differenced, averaged, etc., unless necesssary.
   http://nco.sf.net/nco.html#prc_xcp
6. ncap2 now accepts "sum" as a synonym for "total" in its methods. Additionally, ncap2 has a new function, ncap_stats_wvariance(var,wgt) to computed weighted standard deviations where missing values may be present. Additionally, ncap2 now builds with GSL 2.x.
   http://nco.sf.net/nco.html#ncap2

Additional details are available in the ChangeLog.