Version 4.6.7 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.
The NCO project is coordinated by Professor Charlie Zender of the Department of Earth System Science, University of California, Irvine. More information about the project, along with binary and source downloads, are available on the SourceForge project page.
From the release message:
The main new features in 4.6.7 are in the regridder, which
introduces a sub-grid regridding option, more automated handling
and flexibility with regional grids, and easier flag specification.
ncap2 added specialty functions to simplify handling of missing
values and in-filling large missing regions with interpolated data.
ncatted received an important bugfix for netCDF4 with _FillValue.
PLEASE READ THIS IF YOU USE NCKS TO PRINT DATA:
The ncks default printing will change to CDL in summer 2017.
For 20+ years ncks has, by default, dumped the text representation
of a file in (what we call) "traditional" mode, which best facilitates
meticulous data examination in line-by-line format.
ncdump produces CDL format that is more useful for most NCO users. ncdump has always printed clean CDL and for many years there was
little point in defaulting ncks printed output to CDL.
However, ncks CDL mode now rivals ncdump in many ways.
In particular, ncks --cln now prints times as human-readable calendar
dates, the last ncdump feature that I used which ncks lacked.
Hence in summer 2017 ncks default printed output will change to CDL.
Then one will type simply "ncks" instead of "ncks --cdl".
We just added a new "--trd" option to print traditional output.
Use --trd in your scripts and their behavior will not change.
Otherwise your printing scripts will start to print CDL this summer.
You have been warned.
New Features
-
ncremaphas a new sub-gridscale (SGS) mode,-P sgs. In sub-grid mode, ncremap performs substantial pre- and post-processing so that regridding conserves fields that may represent only a fraction of the entire gridcell. The sub-gridscale fraction represented by each field is contained in variable whose can be set with the new--sgs_frcoption (default is "landfrac"). SGS mode eases regridding of datasets (e.g., from ALM, CLM, CICE) that output data normalized to a fraction (rather than the entire extent) of the gridcell. SGS mode automatically derives new binary masks (--sgs_msk) and allows for additional normalization (--sgs_nrm). Specific flavors of SGS can be selected (-P alm,clm, orcice). These ensure regridded datasets recreate the idiosyncratic units (e.g., %, km2) employed by raw ALM, CLM, and CICE model output. Discerning these regridded from raw datasets is difficult.
ncremap -P sgs -s src_grd.nc -i in.nc -d 1x1.nc -o rgr.nc ncremap -P alm -s src_grd.nc -i in.nc -d 1x1.nc -o rgr.nc ncremap -P cice -s src_grd.nc -i in.nc -d 1x1.nc -o rgr.nc ncremap -P sgs --sgs_frc=aice --sgs_msk=tmask --src_nrm=100 ...
http://nco.sf.net/nco.html#sgs
Thanks to Forrest Hoffmann and Min Xu for prototyping the algorithm and verifying the results. -
ncremapnow automatically indicates to the ERWG weight generator when either or both source and destination grids are regional. This information can considerably reduce the time and space the weight generator requires to generate and store the map-file.ncremapuses fuzzy logic to determine this for inferred grids. The user may informncremapwhether supplied grids (i.e., specified with-sor-g) are regional with the new--rgn_dstand--rgn_srcflags:
ncremap -i in.nc -d dst.nc -m map.nc # Inferred (no option necessary) ncremap -s grd_src.nc -g grd_dst.nc -m map.nc --rgn_src # Supplied
Users may notice significantly faster throughput due to this.
http://nco.sf.net/nco.html#rgn_src -
ncremapswitches have been "rationalized" to eliminate poor UI decisions made in early development. Please read the old/new option meanings and change your scripts accordingly. Now-asubsumes the old role of-w. HINT: use long-options (e.g.,--algorithm=conserve) to future-proof your scripts.
Meaning of-a:
Old: Interpolation algorithm for ESMF only
New: Interpolation algorithm for ESMF and Tempest ('tempest')
Meaning of-E:
Old: Options forESMF_RegridWeightGen(now use-W)
New: Unused
Meaning of-T:
Old: Options forTempestRemap(now use-W)
New: temporary directory (replaces old-U)
Meaning of-U:
Old: temporary directory
New: unpack flag (tellsncremapto unpack data before regridding)
Meaning of-w:
Old:Weight-generatorkeyword. Valid options were 'esmf' and 'tempest'.
New:Weight-generatorcommand string (e.g.,"mpirun ESMF_RegridWeightGen -n 24") Supercedes default'ESMF_RegridWeightGen'or'GenerateOfflineMap'
Meaning of-W:
Old: Unused
New: Options forweight-generator(replaces old -E and -T)
http://nco.sf.net/nco.html#ncremap -
ncap2has a new function namedmissing(), akamask_miss()that returns an array of the same size of its argument, with values of zero where valid data are present and one where missing data are present. The resulting array can be useful in characterizing and masking the original array.
ncap2 -s 'mss_val=three_dmn_var_dbl.missing().ttl($time)' in.nc out.nc
Thanks to Adrian Tompkins for suggesting this feature.
http://nco.sf.net/nco.html#mask_miss -
ncap2now propagates existing chunking parameters from the input file to the output file, and similarly chunks created variables. The rest of NCO has long implemented comprehensive chunking features and nowncap2supports the basic methods.
ncap2 -s 'avg=one_dmn_rec_var.avg()' ~/nco/data/in.nc ~/foo.nc
http://nco.sf.net/nco.html#cll_mth -
ncksnow has a--trd(or--traditional) option to print output in the "tradiational" format. Start to use this unless you prefer CDL output which will become the default in summer 2017, possibly in NCO 4.7.0. At that time, the--cdloption will become default.
Additional details are available in the ChangeLog.
