[netcdfgroup] netCDF Operators NCO version 4.4.3 are ready

The netCDF Operators NCO version 4.4.3 are ready.

http://nco.sf.net (Homepage)
http://dust.ess.uci.edu/nco (Homepage "mirror")

This release is mainly a bugfix and stability release.
The new features may interest those using hierarchical group files,
those needing Python wrappers for NCO, and those accessing strided data
over DAP. Downstream NCO builders will notice many fewer regressions
when building NCO on "old" (pre 4.3.1) netCDF libraries.

Work on NCO 4.4.4 is underway, still focused on stability and speed.
There will be more netCDF4 mop-up, improved group broadcasting
algorithms, and, possibly, improved HDF4 support, and new chunking
features.

Enjoy,
Charlie

"New stuff" in 4.4.3 summary (full details always in ChangeLog):

NEW FEATURES:

A. Group broadcasting works with ensembles. Now files containing
   ensembles (sets of replicated experiments where all ensemble
   members stored in groups residing under a common group) may be
   used in binary operations (i.e., ncbo) with "scalar files",
   i.e., files with the same variables but without ensembles.
   This is useful, for example, in comparing multiple model results
   to a single set of observations.
   ncbo cmip5.nc obs.nc -p ~/nco/data ~/foo.nc
   http://nco.sf.net/nco.html#grp_brd

A. Contributed Python wrappers: Check-out NCOpy by Joe Hamman
   http://nco.sf.net#python

BUG FIXES:

A. NCO now employs a much faster access method for hyperslabs.
   The new method dramatically speeds-up access to strided hyperslabs
   across DAP connections. Thanks to Rich Signell, Jeff Whittaker, and
   Anil for help identifying the problem and the solution.

B. ncwa and ncpdq could sometimes process (i.e., average, permute
   or pack) over dimensions besides those specified by the user.
   This only occurred if the specified dimension name was a proper
   subset of the name of another dimension. In that case, both would
   be processed. Reported by Marie-Alice Foujols.

C. Fix a problem where cell_methods could be written multiple times in
   ensemble datasets.

D. Segregate regression tests so there are fewer failures on older
   netCDF libraries.

KNOWN ISSUES NOT YET FIXED:

   This section of ANNOUNCE reports and reminds users of the
   existence and severity of known, not yet fixed, problems.
   These problems occur with NCO 4.4.3 built/tested with netCDF
   4.3.2-rc1 on top of HDF5 hdf5-1.8.12 with these methods:

   cd ~/nco;./configure --enable-netcdf4  # Configure mechanism -or-
   cd ~/nco/bld;make dir;make allinone # Old Makefile mechanism

A. NOT YET FIXED (NCO problem)
Correctly read arrays of NC_STRING with embedded delimiters in ncatted arguments

   Demonstration:
ncatted -D 5 -O -a new_string_att,att_var,c,sng,"list","of","str,ings" ~/nco/data/in_4.nc ~/foo.nc
   ncks -m -C -v att_var ~/foo.nc

   20130724: Verified problem still exists
   TODO nco1102
   Cause: NCO parsing of ncatted arguments is not sophisticated
   enough to handle arrays of NC_STRINGS with embedded delimiters.

B. NOT YET FIXED (netCDF library problem)
   Probe hidden attributes (chunking, compression) of HDF4 files

   Demonstration:
   ncdump -h -s ~/nco/data/hdf.hdf # (dies)
   ncks -m ~/nco/data/hdf.hdf # (works by avoiding fatal calls)

   20131230: Verified problem still exists

   Cause: some libnetCDF library functions fail on HDF4 file inquiries.
Bug report filed: netCDF #HZY-708311 ncdump/netCDF4 segfaults probing HDF4 file
   Tracking tickets NCF-272, NCF-273

C. NOT YET FIXED (would require DAP protocol change?)
   Unable to retrieve contents of variables including period '.' in name
   Periods are legal characters in netCDF variable names.
   Metadata are returned successfully, data are not.
   DAP non-transparency: Works locally, fails through DAP server.

   Demonstration:
ncks -O -C -D 3 -v var_nm.dot -p http://thredds-test.ucar.edu/thredds/dodsC/testdods in.nc # Fails to find variable

   20130724: Verified problem still exists.
   Stopped testing because inclusion of var_nm.dot broke all test scripts.
NB: Hard to fix since DAP interprets '.' as structure delimiter in HTTP query string.

   Bug report filed: https://www.unidata.ucar.edu/jira/browse/NCF-47

D. NOT YET FIXED (would require DAP protocol change)
   Correctly read scalar characters over DAP.
   DAP non-transparency: Works locally, fails through DAP server.
   Problem, IMHO, is with DAP definition/protocol

   Demonstration:
ncks -O -D 1 -H -C -m --md5_dgs -v md5_a -p http://thredds-test.ucar.edu/thredds/dodsC/testdods in.nc

   20120801: Verified problem still exists
   Bug report not filed
   Cause: DAP translates scalar characters into 64-element (this
   dimension is user-configurable, but still...), NUL-terminated
   strings so MD5 agreement fails

"Sticky" reminders:

A. Pre-built, up-to-date Debian Sid & Ubuntu packages:
   http://nco.sf.net#debian

B. Pre-built Fedora and CentOS RPMs:
   http://nco.sf.net#rpm

C. Pre-built Windows (native) and Cygwin binaries:
   http://nco.sf.net#windows

D. Pre-built AIX binaries:
   http://nco.sf.net#aix

E. NCO support for netCDF4 features is tracked at

   http://nco.sf.net/nco.html#nco4

NCO supports netCDF4 atomic data types, compression, chunking, and groups.

F. Reminder that NCO works on most HDF4 and HDF5 datasets, e.g.,
   HDF4: AMSR MERRA MODIS ...
   HDF5: GLAS ICESat Mabel SBUV ...
   HDF-EOS5: AURA HIRDLS OMI ...

--
Charlie Zender, Earth System Sci. & Computer Sci.
University of California, Irvine 949-891-2429 )'(



  • 2014 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: