As I am currently immersed in Web-service-think (particularly WSDL,
which seems to basically be a more generalized version of what THREDDS
catalog is attempting for scientific data services) I would propose that
the principle of orthogonality might be a useful tool for deciding on
For an XML design, orthogonality means whether or not a given tag or
attribute represents a distinct concept that is in no case expressible
using existing tags and attributes.
While a completely orthogonal tag set results in somewhat less succinct
documents than an approach which defines a number of "special case"
tags, the advantage is that it yields the maximum ratio of
expressiveness to schema complexity.
WSDL is an extreme example of this approach. Unlike them, we may want to
define special-case tags for very common cases to make the
actual documents less unwieldly.
The purpose of this message is not to suggest exactly which cases these
might be; rather, I am just suggesting we take a look the current and
proposed DTD down from this perspective.
A summary of the completely orthogonal concepts I believe we have
introduced thus far (not necessarily named the same as the tags that
currently express the concept):
service type - a named mechanism for accessing scientific data
access - a (named?) binding of a URI to a specific service type
metadata type - a named convention for description of scientific data
metadata - a (named?) binding of a text fragment to a specific metadata type
metadata reference - a (named?) binding of a URI to a specific metadata type
dataset - a named collection of access objects and metadata
collection - a named collection of datasets
collection reference - the URI of a THREDDS XML document containing a
In contrast, the following concepts are not clearly orthogonal to me:
service path, server path, collection path, dataset path, suffix - since
they are only used in the context of the access object, they don't
really add meaning - any catalog using these attributes is equivalent to
one without them, which uses absolute uri's for all of its access objects
compound service / service list - this also doesn't strictly speaking
add meaning since services are only used in the context of access
objects - thus, one access with a compound service type is functionally
equivalent to n access objects with simple service types.
service subtype - unless the values for this attribute are given
standard meanings, this is equivalent to a named access object. even if
the values do have standard meanings, there still seems to be some
overlap with metadata type.
catalog, server - if you factor out the path attribute, these are
equivalent to collections
documentation - equivalent to metadata with a human-readable metadata type
document - same as documentation, except with the connotation that it is
not critical to interpreting the dataset
One that I am not sure about is the "attribute" tag, since I am not
clear on how this is intended to be used. Is it for the THREDDS parser,
or passed directly to the user? Will there be standardized names and
values for attributes?
A reminder, I am not trying to say specifically whether any of these
tags should be kept or dropped. I am merely suggesting that we might
want to focus on tags and attributes that represent orthogonal concepts,
and be a bit more choosy about the rest.
Also, I would suggest that any proposed extensions that *are* genuinely
orthogonal to the original tag set (although I'm not sure we've had any
thus far) be given special consideration, since by definition, there is
no workaround if they are not included.
John, hope this is useful input.
Benno Blumenthal wrote:
John Caron wrote:
Im trying to think what is the meaning of serviceType="Catalog" in
What should the client assume? It seems that if you want the client
able to get the collection as a dataset, then you add a dataset
you want the client to "drill down" further, then a collection or
element can do that. What I have removed is the clear association
the dataset element and the collection, eg that these are the same
have also made it more cumbersome(need two elements). I agree these are
The client does not know that these are two different ways of looking at
the same thing -- the key piece of information that was trying to be
conveyed. The client does not have to present both -- maybe the client
only presents THREDDS choices because it has no DODS capabilities,
another client does not present the drill-down because it does have
The fact that you can produce a dataset as COARDS vs DIF, etc is
also for me
not so great of an example. Rather than modifying the underlying
(eg DODS), it seems simpler to add a metadata element. I admit that
just an idea which has not been done yet. And you already have a
does in fact modify the data access. But think of it from a client POV.
Should she search through the services looking for a service of type
subtype COARDS? Or search through the metadata looking for COARDS
independent of service type?
My point was your metadata tag was services for metadata. clients that
can only handle COARDS metadata would ask for COARDS metadata services.
A more compelling example would be where the dataset is served up
FTP and DODS, and ADDE, etc. But then I wonder/doubt whether one URL is
likely to be able to be used for all these services.
DODS already has multiple services -- ascii is not necessarily present,
some of the selection interfaces are optional, metadata is optional.
> I am also concerned about XYZT clients (4D world view) -- how
> them against higher (and other) dimensional data (ensemble member
> spectral, different kinds of time (forecast start, lead, target
> could convert to multiple datasets, or spatial grids, but it
would be nice
> advertise the service. As well as supporting various binary and
> formats. Or the THREDDS dataset (as opposed to collection/catalog)
I am not clear of "protect", did you mean "project" ?
Protect -- 4D world clients simply fail when given something else, I
would like them to have an alternative.
> 2) The access for the dataset LEVITUS94 is again via THREDDS
> collection) or via DODS (the access statement). Adding another
> the collection called "Daily" is not the same meaning at all.
Sorry, I should have had:
<dataset name="LEVITUS94 dataset" urlPath="SOURCES/.LEVITUS94/dods"/>
<catalogRef xlink:title="Drill down into dataset"
In this case the dataset is presented to the user for immediate
AND a link is presented for drilling further down.
This is the wrong example: LEVITUS94 was a collection that was also
available through DODS -- now you have lost it completely.
It is an example for the subdatasets ANNUAL, etc, with the flaw that
clients no longer can tell that these are two different ways of getting
the same thing as mentioned above.
Besides, multiple services also will show up in aggregations of
THREDDS catalogs -- multiple servers serving the same dataset could be
represented as a single entry with multiple services -- in this case,
services with identical attributes except for path information.
The main thing service does is to let you specify a type and factor
common URL base. then this is passed to "protocol aware" code.
das,dds,dods,info,ascii subservice URLs are always regular in how
formed, it seems unnecessary to actually specify them. In principle
subservices are probably useful but some concrete examples are needed.
As I mentioned earlier, not all DODS servers have all the services.
Even if they did, it would not hurt to be able to list them.
While I have not given examples, different datasets will have
> services, which is why I kept specifying using the access tag.
> will be incompatible with certain representations, so the service
> vary. One could argue that the THREDDS standard collection is a
> available via DODS -- certainly that is the case for me.
I understand you want to compactly specify what services are
datasets. Im not sure we have enough examples to make sure we are
right. I am also oriented towards incremental design, doing what we
right and iterating.
OK with me, but we have lost the structure I was trying to express --
alternate ways of accessing the same object, with emphasis on the same
Dr. M. Benno Blumenthal benno@xxxxxxxxxxxxxxxx
International Research Institute for climate prediction
Lamont-Doherty Earth Observatory of Columbia University
Palisades NY 10964-8000 (845) 680-4450
joew@xxxxxxxxxxxxx / (707)826-2631
Center for Ocean-Land-Atmosphere Studies (COLA)
Institute for Global Environment and Society (IGES)