The netCDF operators NCO version 4.0.1 are ready.
http://nco.sf.net (Homepage)
http://dust.ess.uci.edu/nco (Homepage "mirror")
This version offers many minor fixes and improvements.
Short explanations of these appear below.
Work on NCO 4.0.1 is underway. Areas of improvement include
Support for spectral transforms, chunking in all operators,
more chunking rulesets.
"New stuff" in 4.0.1 details:
A. Add ncks --fix_rec_dmn switch to change all record dimensions in
in.nc into fixed dimensions in out.nc:
ncks --fix_rec_dmn in.nc out.nc
B. Add ncks --mk_rec_dmn dmn_nm switch to define the indicated dimension
as the record dimension. The specified dimension should be the
first dimension, not an inner dimension, of some variable.
Otherwise ncks will fail with an error.
ncks --mk_rec_dmn time in.nc out.nc
C. Fix problems converting netcdf4_classic input files to netcdf3
ncks -3 in_4c.nc out_3.nc
D. Fix behavior of ncrename when target (variable/dimension/attribute
does not exist and name was prefixed with '.'. Used to die messily.
Correct behavior, now implemented is to skip it with a warning.
Thanks to Patrick for reporting this bug.
E. Configure: --enable-netcdf-4 is synonymous with --enable-netcdf4
Now NCO configure accepts the usage of the netCDF configure.
According to the Unidata netCDF team, the converse is also true.
F. ncks warns when printing packed contents of packed variables
G. Fix misleading ncbo WARNING about non-duplicated variables
H. Known Problem:
Namespace collision between mvapich2, netCDF4, and NCO when netCDF
(4.1) is compiled with parallel HDF5 (1.8.4). Thanks to anonymous
for pointing this out. We are working on it!
"Sticky" reminders:
J. ncks and ncecat support netCDF4 chunking options:
These options can improve performance on large datasets.
Large file users: Send us suggestions on useful chunking patterns!
More useful chunking patterns may be implemented in NCO 4.0.1.
ncks -O -4 --cnk_plc=all in.nc out.nc
http://nco.sf.net/nco.html#chunking
K. Pre-built, up-to-date Debian Sid & Ubuntu Karmic packages:
http://nco.sf.net#debian
L. Pre-built Fedora and CentOS RPMs:
http://nco.sf.net#rpm
M. Did you try SWAMP (Script Workflow Analysis for MultiProcessing)?
SWAMP efficiently schedules/executes NCO scripts on remote servers:
http://swamp.googlecode.com
SWAMP can work command-line operator analysis scripts besides NCO.
If you must transfer lots of data from a server to your client
before you analyze it, then SWAMP will likely speed things up.
N. NCO support for netCDF4 features is tracked at
http://nco.sf.net/nco.html#nco4
NCO supports netCDF4 atomic data types, compression, and chunking.
NCO 4.0.1 with was built and tested with HDF5 hdf5-1.8.4 and with
netCDF4 netcdf-4.1-snapshot2009111500.
NCO may not build with earlier, but should build with later, netCDF4
releases.
This is particularly true since NCO 4.0.1 takes advantage of
an internal change to the netCDF API in June 2009 for
nc_def_var_chunking().
export NETCDF4_ROOT=/usr/local/netcdf4 # Set netCDF4 location
cd ~/nco;./configure --enable-netcdf4 # Configure mechanism -or-
cd ~/nco/bld;./make NETCDF4=Y allinone # Old Makefile mechanism
O. Have you seen the NCO logo candidates by Tony Freeman, Rich
Signell, Rob Hetland, and Andrea Cimatoribus?
http://nco.sf.net
Tell us what you think...
Enjoy,
Charlie
--
Charlie Zender, Department of Earth System Science
University of California, Irvine (949) 891-2429 :)
