Hi,
I would anticipate that the portion of Steve's netCDF adapter which
constructs the MathType--through intelligent guessing--could be
reused regardless of whether the underlying access is built on DODS
classes or netCDF classes.
It might take more human effort to build on the DODS classes, but
that solution makes more data accessible to VisAD users. For example,
there are DODS servers that make data available only as "sequences"
and "structures" in DODS parlance, and I think these data are
essentially invisible through the (DODS-implemented) netCDF API. The
VisAD data model is rich enough to make sense of these data types,
but the netCDF API would block access to them (unless I am mistaken).
Hence, I favor option two (building a VisAD adapter directly on the
DODS classes) unless 1) I have misunderstood the limitations of the
netCDF interface or 2) this path is substantially more expensive in
human effort.
Though John's message seems slightly tangential to Ethan's question
of how best to link VisAD applications with DODS servers, the message
is quite important to Unidata and DODS in a larger sense. I see
John's view as consistent with the objective we articulated in the
"Unidata 2003" proposal to NSF as striving to build a framework
(http://www.unidata.ucar.edu/proposals/NSF2003/Geosif.html) having:
"1. Objects that hold data and behave like sampled functions on a
specified domain, with methods specialized for irregular and
gridded data (with images included in the latter). Also, methods
for storing and retrieving portable copies of these objects,
partially or wholly compatible with netCDF. (These generalize
netCDF, because an array may be viewed as a function whose domain
is a set of index vectors. Sampled functions can represent any
dataset of observations or state descriptors, with navigations
and calibrations.)
"2. Objects that represent the (potentially continuous and infinite)
domains of sampled functions, including complex shapes as for
representing terrain, ocean basins, rivers, etc. Also, an
associated and extensible set of space-time coordinates,
encompassing various models for the shape of the earth as well as
state-dependant vertical coordinates, such as pressure, density,
potential temperature, etc."
I think this objective remains valid, and perhaps "pluggable semantic
parsers" could help us get there, though I don't yet understand
exactly what it means. I once proposed that Steve's netCDF adapter
for VisAD could be augmented with a constructor whose signature
included a text string making the MathType explicit, i.e., replacing
Steve's intelligent guesswork with knowledge of specific netCDF files
or conventions. Is the pluggable semantic parser idea more or less
equivalent?
Underlying any parser, I suspect, must be a data model that is rich
enough to embrace the full complexity of the data representations and
the data-analysis functions we face. Therefore, I propose that we
consider the following question:
Can we agree on the VisAD data model--perhaps accompanied with
some concept of pluggable parsers (and a few examples for the
most common netCDF and JGOFS conventions) to improve the
understandability of the model--as a foundation for the higher
level of semantics that now should be embedded in the data sets
and data servers we deal with in Unidata and DODS?
If this is not the right question to pose, I welcome alternatives.
Dave
--On Thursday, 13 July2000 22:36 -0600 John Caron
<caron@xxxxxxxxxxxxxxxx> wrote:
tomw wrote:
Ethan:
> We've thought more about the first option than the second
> because: option one would also be valuable outside the VisAD
> community; and with option two, the use of attribute conventions
> would have to be added instead of taking advantage of the netCDF
> conventions already in the VisAD netCDF adaptor.
I agree about the first option. As long as VisAD uses the Java
NetCDF library, and DODS provides the mechanism to extend these
classes with ones which would, when needed use DODS access, it
would allow anyone with an app that uses NetCDF to use DODS --
just like in the C and Fortran world, right? That would seem to
be the all-around best solution.
(One of the reasons I would not want to make a DODS VisAD adapter
is that there is a lot of 'magic' (read: hard work) that Steve
Emmerson did in the NetCDF adapter to turn the metadata into the
proper MathTypes in VisAD and make Data objects that are directly
usable. It would seem the same kind of work would be needed to
convert a DODS data object, as well.)
Does someone see this differently?
well of course i see it differently :^}
It seems to me that the problem is that the netcdf API is not high
level enough for what we want to acheive (eg create visad objects
with semantics that match the data). I suspect DODs isnt either,
but im not sure. i dont see how you can adequately represent the
data semantics unless you are parsing something like a netcdf
Convention, but perhaps I'm missing something. it seems like the
problem must be compounded if you try to map DODS to netcdf
(partly because the two APIs have some impedence mismatch).
One solution that keeps coming up for me is to create pluggable
semantic parsers specific to various datasets and/or Conventions.
Also create C/ Fortran/Java data APIs for the data _writers_, and
get away from specifying things at the netcdf attribute level (we
will still need that, but most users would prefer a higher level
API).
I plan on looking at DODS (and ADDE) more closely so I can
understand them better with respect to these issues. Any pointers
appreciated.
=======================================
Dave Fulker
Unidata Program Director
University Corp for Atmospheric Research
----------------------------------------
Mail: PO Box 3000
Boulder, CO 80307-3000 USA
Email: fulker@xxxxxxxxxxxxxxxx
Phone: 303-497-8650 Fax: 303-497-8690
=======================================
