[no subject]

• To: jwlong@xxxxxxxxxxxxxxxxx
• From: Russ Rew <russ@xxxxxxxxxxxxxxxx>
• Date: Wed, 6 May 1992 11:33:28 -0600

Jeff,

Thanks for sending the description of SILO.  In general, I like your
extensions to netCDF, and the way you managed to keep the new interface
compatible with existing netCDF applications.  I've provided tentative
answers to your questions below, after some comments and questions of my
own.  If it's more convenient for you to just send me a copy of the detailed
document rather than comment on all of these by email, that would be fine.

First, it appears to me that the "directory" and "object" extensions are
independent, and that the directory extensions are more fundamental.  If
object names are globally unique rather than scoped to directories (I
couldn't determine this for sure from your description), than an object
containing other components could be represented by a global attribute with
a string value that listed the component dimensions, variables, and objects.
For component names that differed from actual names, a syntax convention
could be chosen to specify both names, e.g. including the component name in
parentheses after the actual name.  An attribute convention could be adopted
that all such object attributes should have conventional names, for example
beginning with the characters "Group_".  Unless I'm missing something, I
think it would be possible to write your object extensions as a small
library that assumed such a set of attribute conventions.  I'm not asserting
this would be a better way to do it, only that it appears to be a possible
implementation that wouldn't require interface extensions, only a new layer
on top of the existing interface, and agreement about the attribute
conventions.

I'm convinced that adding the directory primitive requires the kinds of
additions to the interface you have proposed.  Directories would make some
things we are proposing to do in the netCDF operators simpler, for example
merging several input netCDF datasets into one output netCDF, but they also
add some complexity to the interface.  Since you've made the additions in an
upward-compatible way, it seems that users could ignore this added
complexity if they don't need the benefits it provides.

If you don't create any directories, what name do you get back when you
inquire about the name of the current directory?  Is there a default
directory id and name even when no calls to ncdirdef() have occurred?

You wrote that there is still just one unlimited dimension ID.  Is it
possible to have multiple unlimited dimensions in different directories, but
with the same ID?

What is the "type" of an "object" that contains heterogeneous components,
such as dimensions, variables, and other objects?  Are object ID's unique
within a dataset, like dimension ID's, or just within directories, like
variables?  Have you had a need for interfaces to add a new component to an
existing object, to delete a component from an object, to copy or move a
component from one object to another, or to iterate through the components
of an object?  Can the same component be in multiple objects?  If objects
were constructed with an "add" function, then you wouldn't have to specify
the component type, but could instead use something like ncadddim() and
ncaddvar() to add dimensions and variables to an object.

Is it only possible to create directories and/or objects in "define mode"?
Did you come up with an implementation that doesn't require distinguishing
between "define mode" and "data mode"?

Have you defined a Fortran interface for the new C functions in the
interface?

I notice you aren't on the netcdfgroup mailing list, so you may not know
about the changes we made from int to long for the types of dimensions and
hyperslab indices in netCDF 2.0.  Is SILO based on netCDF 2.0 or an earlier
version? 

Is the code for SILO available?  Does your implementation require a
commercial or proprietary database underneath SILO?

Now here's my tentative answers to your questions:

> 1. Should the concept of relative IDs be kept, even though it makes it more
>    difficult to specify object components? Should absolute IDs be
>    introduced, even though this departs from the netCDF model? It could be
>    possible to have both schemes simultaneously, and provide a mechanism
>    for mapping between absolute and relative.

  I think you made the right decision to change to relative IDs.  With
  absolute IDs, you would need to add more functions to iterate through all
  the variables in a directory, for example.

  With relative IDs, when you are in a SILO directory you apparently cannot
  refer to components of other directories, but must always change
  directories first.  With absolute IDs, It might be possible to get a
  dimension or variable from the closest ancestor directory that contains it
  if it is not found in the current directory.  This would allow you to
  "factor out" common data without having to know exactly which parent
  directory contains it and to insert new intermediate levels in a directory
  hierarchy without breaking existing applications, so directories could be
  used for inheritance as well as name scope.  I don't think this would be
  useful enough to justify changing from relative to absolute IDs.

> 2. Is there a better name for objects than 'objects'? Perhaps 'groups'?

  "Object" has so many other connotations these days, that I would prefer a
  more specific name that had the connotation of being a collection.
  "Group" is fine, and some other possibilities include container, package,
  bin, bundle, etc.

I'm CC:ing this reply to a local mailing list, netcdf-hdf@xxxxxxxxxxxxxxxx,
of Unidata staff members interested in the NCSA project to put the netCDF
(or SILO) interface on top of HDF.

----
Russ Rew                                        russ@xxxxxxxxxxxxxxxx
Unidata Program Center / U.C.A.R.
P.O. Box 3000
Boulder, Colorado 80307-3000