The netCDF operators NCO version 4.0.7 are ready.
http://nco.sf.net (Homepage)
http://dust.ess.uci.edu/nco (Homepage "mirror")
This release does one thing only, it fixes an important bug:
http://nco.sf.net#bug_hyp_dgn
Now NCO returns correct hyperslabs for hyperslabs or bounding boxes,
specified with -d or -X, respectively, that do not include at least
one point. Formerly, NCO could erroneously return all data on such
hyperslabs. Thanks to Jeff Daily for reporting this. Although most
users do not specify empty hyperslabs, we urge all users to upgrade to
NCO 4.0.7+ just to be safe.
Also worth mentioning again is that the latest Debian (Sid) and Ubuntu
(Maverick) NCO packages support DAP, netCDF4, and UDUnits2.
http://nco.sf.net#debian
http://nco.sf.net#ubuntu
Work on NCO 4.0.8 is underway, focused on issues described in the
KNOWN BUGS NOT YET FIXED section below.
Enjoy,
Charlie
Other "New stuff" in 4.0.7 summary (full details always in ChangeLog):
A. None
KNOWN BUGS NOT YET FIXED:
This section of the ANNOUNCE file is intended to make clear the
existence and severity of known, and not yet fixed, problems.
A. NOT YET FIXED
Correctly handle stride argument when accessing some files through
some DAP servers.
DAP non-transparency: Works locally, fails through DAP server.
Demonstration:
ncks -O -F -D 9 -v weasdsfc -d time,100,110,5
http://nomad3.ncep.noaa.gov:9090/dods/reanalyses/reanalysis-2/6hr/flx/flx
~/foo.nc
# Fails as of 20110114, old DAP server?
ncks -O -F -D 9 -v three_dmn_rec_var -d time,1,10,5
http://motherlode.ucar.edu:8080/thredds/dodsC/testdods/in.nc ~/foo.nc #
Works as of 20110114
20110114: Verified problem still exists on NOAA DAP server
Bug report not filed: Suggest users upgrade DAP servers?
B. NOT YET FIXED
Correctly handle NC_CHAR coordinates (like fl_dmn) on DAP files.
DAP non-transparency: Works locally, fails through DAP server.
Demonstration:
ncks -D 4 -v fl_dmn -O
http://motherlode.ucar.edu:8080/thredds/dodsC/testdods/in.nc ~/foo.nc
20110114: Fixed in Unidata DAP server? Need confirmation works elsewhere
Bug report filed: netCDF #JZN-949284 NC_CHAR coordinate via DAP
C. NOT YET FIXED
Correctly average NC_CHAR coordinates from DAP files.
DAP non-transparency: Works locally, fails through DAP server.
Demonstration:
ncra -O -C -D 3 -p
http://motherlode.ucar.edu:8080/thredds/dodsC/testdods in.nc in.nc
~/foo.nc # Breaks with "NetCDF: DAP server error"
20110221: Verified problem still exists
Bug report filed: netCDF #JZN-949284 NC_CHAR coordinate via DAP
D. NOT YET FIXED
Correctly read netCDF4 input over DAP, write netCDF4 output, then
read resulting file.
Replacing netCDF4 with netCDF3 in either location of preceding
sentence leads to success.
DAP non-transparency: Works locally, fails through DAP server.
Demonstration:
ncks -4 -O -v three_dmn_rec_var
http://motherlode.ucar.edu:8080/thredds/dodsC/testdods/in_4.nc ~/foo.nc
ncks ~/foo.nc # breaks with "NetCDF: Invalid dimension ID or name"
20110114: Verified problem still exists
Bug report filed: netCDF #QUN-641037: dimension ID ordering assumptions
"Sticky" reminders:
A. All operators support netCDF4 chunking options:
These options can improve performance on large datasets.
Large file users: Send us suggestions on useful chunking patterns!
ncks -O -4 --cnk_plc=all in.nc out.nc
http://nco.sf.net/nco.html#chunking
B. Pre-built, up-to-date Debian Sid & Ubuntu Lucid packages:
http://nco.sf.net#debian
C. Pre-built Fedora and CentOS RPMs:
http://nco.sf.net#rpm
D. Did you try SWAMP (Script Workflow Analysis for MultiProcessing)?
SWAMP efficiently schedules/executes NCO scripts on remote servers:
http://swamp.googlecode.com
SWAMP can work command-line operator analysis scripts besides NCO.
If you must transfer lots of data from a server to your client
before you analyze it, then SWAMP will likely speed things up.
E. NCO support for netCDF4 features is tracked at
http://nco.sf.net/nco.html#nco4
NCO supports netCDF4 atomic data types, compression, and chunking.
NCO 4.0.7 with was built and tested with HDF5 hdf5-1.8.4-patch1 and with
4.1.2-beta2-snapshot2010101108
NCO may not build with earlier, and should build with later, netCDF4
releases.
This is particularly true since NCO 4.0.7 takes advantage of an
internal change to the netCDF nc_def_var_chunking() API in June 2009.
export NETCDF4_ROOT=/usr/local/netcdf4 # Set netCDF4 location
cd ~/nco;./configure --enable-netcdf4 # Configure mechanism -or-
cd ~/nco/bld;make dir;make all;make ncap2 # Old Makefile mechanism
F. Have you seen the NCO logo candidates by Tony Freeman, Rich
Signell, Rob Hetland, and Andrea Cimatoribus?
http://nco.sf.net
Tell us what you think...
--
Charlie Zender, Department of Earth System Science
University of California, Irvine (949) 891-2429 :)
