The netCDF operators NCO version 2.2.0 are ready.
Changes since version 2.0.0 include bugfixes and a
working (and powerful!) ncap implementation.
ncap is the netCDF arithmetic processor, an operator
which carries out arbitrary arithmetic commands on
netCDF variables and attributes using short command-line
instructions or long scripts as input.
In the fine tradition of beta quality software, ncap
is completely undocumented but see
nco/data/ncap.in for a sample run script.
NCO version >= 2.0.1 requires netCDF version >= 3.5.
Get the new version of NCO and read the User's guide on
http://nco.sourceforge.net
I would to acknowledge the tremendous contribution of
Henry Butowsky to NCO. Henry picked up the dormant ncap
project, and did the lion's share of the work in completing it.
Thanks Henry!
Enjoy,
Charlie
P.S. Building ncap requires a good lexer and parser (flex, bison)
and ANSI C implementation with float math functions (e.g., tanf).
For that reason it does not build by default and you must
specifically build ncap with "make ncap".
ncap builds successfully on our IRIX, Solaris, and Linux platforms.
ncap does not yet build on AIX (math float functions do not link).
Summary of changes since 2.0.0:
1. ncap: lexer/parser redefinition, I/O bookkeeping, wrappers for
NCO arithmetic functions, attribute handling, type-specific intrinsic
arithmetic functions, string handling, test script (Butowsky)
2. Fix ncks units printing bug introduced in 2.0 (Zender)
3. Improved version information with, e.g., ncks -r (Zender)
4. Use new NC_NAT token _requires_ netCDF 3.5 (Zender)
5. ncap: global attribute handling, stretching mixed rank
variables, exclusive state lexing, LHS casting (Zender)
6. libnco_c++, a C++ API for libnetcdf (Zender)
A few notes on the objective of ncap:
NCO's objective is to solve everyday needs of recurring tasks
in scientific data analysis without requiring tedious programming.
Previously, NCO had adequate file handling capabilities but very
limited, non-extensible, data processing capabilities.
ncap changes that. ncap's capabilities are between those of a regular
NCO operator and those of a full-featured, high-level, interactive
language (e.g., IDL, Matlab, NCL).
However, now that ncap has a language parser, there is no real limit
to what syntactic features could be added. Of course the
only features that will be added are those the developers care about,
so your feedback is encouraged:
What is the highest priority feature for you?
ncap is best suited now to compute simple derived variables, e.g.,
x=gw*sqrt(u^2+v^2). Hyperslabs (subscripting) are not yet supported,
but broadcasting (unambiguous array expansion) is fully supported,
Also supported is "LHS casting", where the desired dimensions of the
result are simply specified on the LHS of an expression, e.g.,
a[lat,lon,lev]=1.0; // a is created and stored as 3-dimensional array
pressure[time,lat,lon,lev]=P0*hyam+hybm*PS; // No fuss, no muss
To give you the flavor of how ncap may be used for day-to-day tasks,
here are a few commands which illustrate how to add some typical derived
fields to climate model output:
ncap -O -s "wnd=sqrt(wnd_znl^2+wnd_mrd^2)" in.nc in.nc
ncap -O -s "LWCF=FLNTC-FLNT" -s "SWCF=FSNT-FSNTC" in.nc in.nc
ncap -O -s "DSTDPSTTL=DSTDPSTRB+DSTDPSGRV+DSTDPSWET" in.nc in.nc
ncap -O -s "prs[time,lat,lon,lev]=P0*hyam+hybm*PS" -s "dns=prs/(287.05*tpt)" -s
"cnc=mmr*dns" in.nc in.nc
There are undoubtedly small bugs remaining in the parsing
of expressions. We encourage you to stress the parser with complex
expressions and send us any problems you find. Concurrently, we will
steadily extend ncap functionality to include more powerful features.
Some of the most interesting ideas for for new ncap features include:
1. Standard procedural expressions, e.g., "if then else"
2. Fortran-like index-based hyperslab specification, e.g., y=u(:,2:4,:)^2
3. NCO Coordinate value subscripting, e.g., y=u(-90.0:0.0,0.0:180.0)^2
4. Pre-defined functions to access NCO internals, e.g.,
type_convert(variable,"double"), min/max/avg(variable,variable),...
5. Linking with GNU Scientific Library (GSL)
6. filename::variable@attribute notation for accessing multiple files
7. Loop over files performing identical operations on each
8. Console mode for true interactive processing
The list of ncap-specific features/bugs that we consider top
priority appears at the top of the NCO TODO file.
