Hi Tennessee,
On Aug 16, 2005, at 2:54 AM, Tennessee Leeuwenburg wrote:
Peter,
One further question re your 2003 paper. One of the interoperability
layers you identify as "Semantic", and the comment is "more
consistent" -- by this do you mean such things as conventions within
formats, such as COARDS within NetCDF?
Sort of, the structural part of COARDS. I actually identify the layer
that you are referring to as "Semantic-Structure". The idea here is
best understood in terms of the example presented in the paper.
Consider a set of 2 dimensional satellite-derived sea surface
temperature (SST) fields, each field consisting of SST on a grid in,
say the western North Atlantic, at one instant in time. All grids in
the data set are in the same geographic projection. The data set might
have all of the fields, say one per day, for a 10 years. These data
could be stored a number of different ways:
1) 3650 files, each file representing one 2-d field,
2) One 3-d (lat, lon, time) file with the time dimension having 3650
elements,
3) Ten 3-d (lat, lon, time) files with the time dimension of each
having 365 elements,
4) One 3-d (time, lat, lon) file, again with time having 3650
elements,...
The "Structure" layer says that all such data sets should be
represented as one 3-d object. The "Semanti-Structure" layer says that
they should all be represented as one 4-d object with lat always being
the first dimension, lon always being the second dimension, depth the
third and time the fourth. Since this is no depth in this example, the
3 layer would just have one element. The point here is that the
structure is constrained by the semantics. As you move from one layer
to the next, less metadata are required. The "Structure" layer does not
specify the order of the variables, so metadata telling the client what
the order is is required. The Semantic-Structure layer does not require
such metadata. Higher up in the level structure is the metadata that
describes the data, the metadata that I refer to as semantic and
syntactic metadata. I don't think that it is worth making a big deal
out of the Semantic-Structure level, I simply included it to make the
overall picture clear. The amount of metadata required to specify the
semantics of the structure is small. Furthermore, there is not a lot of
agreement on the semantics of the structure which means that data are
stored in a variety of semantic structures. Converting the data on the
fly to a consistent semantic structure is a lot more work than
converting them to a consistent structure, the next level down. I think
that that is valuable and that's what the Aggregation Server does.
Do any such conventions exist for OpenDAP?
No. The idea is to let different disciplines impose whatever structure
they want on top of OPeNDAP. The more appropriate question would be
"Does the oceanographic (or the Earth Science) community have any such
convention?" Unfortunately, the answer is again no. That is my fault. I
should have recommended one a long time ago. I think that COARDS (or
CF) would be an excellent starting point and COARDS is becoming a de
facto standard in that many data providers use it. I was reluctant to
impose COARDS on the oceanographic use of the system since that would
mean the reordering of some archives which could be very costly. I
prefer a modified C
COARDS, one that does not adhere to the semantic structure, but does
adhere to the rest of the standard.
Here at BOM we more or less have our conventions agreed on for my
particular project, but if there were any recommended best practise it
could be interesting.
What do you think of adopting COARDS or CF? Would one or the other
address all of your data sets? If not, what is missing? If it could,
but your would rather not adopt it, it would really be useful to know
why and to know what you guys have adopted and why. I am hoping that
the Marine Metadata effort comes up with a recommendation for the
oceanographic community. I'm not sure what is being done for the
meteorological community, but it would be great if there was a similar
effort (to the Marine Metadata program) and if they came up with a
recommendation.
Peter
Cheers,
-Tennessee
--
Peter Cornillon
Graduate School of Oceanography - Telephone: (401) 874-6283
University of Rhode Island - Fax:
(401) 874-6728
Narragansett, RI 02882 - E-mail:
pcornillon@xxxxxxxxxxx
