Re: CDF, netCDF and HDF

• To: hdf-netcdf@xxxxxxxxxxxxx, lloydt@xxxxxxxxxxxxxx
• Subject: Re: CDF, netCDF and HDF
• From: "Glenn P. Davis" <davis@xxxxxxxxxxxxxxxx>
• Date: Fri, 15 May 1992 17:24:43 -0600

Thanks Glenn for the clarification.  This addresses one of the reasons I
made the posting to the mailgroup -- to get current status on implementations
and current thinking from the actual developers.

I assume by changing shape you mean NOT considering the unlimited dimension.
Obviously, that dimension could be viewed as being part of the definition of
shape for a variable.  It would appear that netCDF allows one to change the
shape along that dimensional axis by adding instances of variables for the rest
of the shape definition without copying.  On the other hand, deleting an
instance (i.e., a record in the conceptual equivalent in the CDF parlance) of
a variable would also change the shape.  Is this supported in netCDF without
copying?  If so, is it done by just tagging the offending element?  Is any
space compression/garbage collection done after repeated such operations
because of the potential of wasted space?  Clearly, a netCDF copy operation
would take care of that, if required.  For large data sets, this could be an
expensive operation.
------------------------------- Referenced Note ---------------------------
Received: from newton.ncsa.uiuc.edu by watson.ibm.com (IBM VM SMTP V2R2)
   with TCP; Fri, 15 May 1992 19:32:56 EDT
Received: from unidata.ucar.edu by newton.ncsa.uiuc.edu with SMTP id AA20586
  (5.65a/IDA-1.4.2 for lloydt@xxxxxxxxxxxxxx); Fri, 15 May 1992 18:24:47 -0500
Received: from zeppo.unidata.ucar.edu by unidata.ucar.edu with SMTP id AA26730
  (5.65c/IDA-1.4.4 for <hdf-netcdf@xxxxxxxxxxxxx>); Fri, 15 May 1992 17:24:43 
-0600
Received: by zeppo.unidata.ucar.edu id AA09547
  (5.65c/IDA-1.4.3 for lloydt@xxxxxxxxxxxxxx); Fri, 15 May 1992 17:24:43 -0600
Message-Id: <199205152324.AA09547@xxxxxxxxxxxxxxxxxxxxxx>
Cc: goucher@xxxxxxxxxxxxxxxxxxxx

> From owner-netcdf-hdf@xxxxxxxxxxxxxxxx Fri May 15 15:48:37 1992
> Date: Fri, 15 May 1992 16:42:13 EDT
> From: "Lloyd A. Treinish" <lloydt@xxxxxxxxxxxxxx>
> To: hdf-netcdf@xxxxxxxxxxxxx
> Cc: goucher@xxxxxxxxxxxxxxxxxxxx
> Subject: CDF, netCDF and HDF
>
> A second issue is data structure residency and how it is supported.  For
> scaling to any reasonably interesting data set by size, structure and breadth
> (i.e., number of parameters/variables/fields), data structures must be disk-
> resident and have a built-in caching mechanism appropriate for those
> structures.  Both CDF and netCDF attempt to do this.  In addition, 
> transaction-
> like operations on data must be supported.  In other words, the ability to
> query, update/modify, delete data in-place is required.  If a substantial
> investment in building a large data set is made, it is too expensive to make
> updates via copying.  If I am current in my knowledge of the netCDF and CDF
> implementations then this is supported in CDF and not in netCDF.  In the HDF
> case none of these ideas apply because the data structures are memory 
> resident.

Well, in netCDF, you can "query, update/modify, delete data in-place" as
required. You can change the value of attributes. What you can't do, without
copying, is change the "shape" of the netCDF once it has been defined.

• Follow-Ups:
  • CDF, netCDF and HDF
    • From: Russ Rew