RE: 4D dimensions and other conventions...

Russ Rew (russ@unidata.ucar.edu)
Tue, 29 Sep 92 09:27:45 -0600 

The formatting of yesterday's message from Steve Hankin to the netcdfgroup
mailing list was inadvertantly corrupted here at Unidata, so I've appended a
reposting of the message in a more readable form.  Let me also take this
opportunity to remind members of the mailing list that administrative
requests for addition to or deletion from the list should be sent to
netcdfgroup-adm@unidata.ucar.edu rather than the full mailing list.  Thanks.

From: HANKIN@noaapmel.gov
Date: Mon, 28 Sep 92 13:42:31 -0800
Subject: RE: 4D dimensions and other conventions... 

Hello Rich, Tim, Ken, et. al.,

I've been following your discussion about netCDF styles with interest as my
own group - a numerical modeling group - shares the similar concerns: how to
use netCDF to achieve a compatible representations of our model data
(gridded, multi-gigabyte, multiple variables on staggered grids) as well as
PMEL's EPIC data (down the hall) and outside institutions, too.

This business of time axis representations is leading us all to similar
solutions. Rich has described a global variable called "base_date" which
"specifies the Gregorian start date".  Similarly, the file
"conventions.info" available from unidata.ucar.edu suggests e.g.

     variables:
         double time(nobs);
         time:units  = "milliseconds since (1992-9-16 10:09:55.3 -600)"

Our own software, FERRET, uses a solution e.g.:

        float TIME(TIME) ;
                TIME:units = "seconds" ;
                TIME:time_origin = "14-JAN-1976 14:00:00" ;

and accepts int, long, float, or double data types.

While all of these are very similar solutions they are also incompatible.
How are time-date strings formatted?  Where should the time origin be
placed: in the units string? in a global attribute?  in a variable
attribute?  If in an attribute, what is the attribute name?  Is the data
type mandated?  Does the axis have to be a "coordinate variable" (dimension
name=variable name) ?  etc. etc.  Similar issues arise for if/how to map
gridded data onto 4-dimensional grids.  Mandatory ordering of axes?
Mandatory axis names?  Mandatory units choices?  What to do with missing
axes (e.g. Z axis of vertically averaged flow)?

It seems to me that if we want to adopt conventions for these issues now is
the time to do it.  NetCDF can fail to be a "standard" in any meaningful way
if these issues are not addressed somewhat formally by "users" (us) acting
as a community.  I have some personal experiences with this type of
standards-failure as a member of the ANSI committee that creates CGM (the
Computer Graphics Metafile).  CGM, a broadly conceived standard, has
expected user communities to develop "profiles" that dictate their
particular style choices and ensure interoperability.  The user communities
have mostly failed to get organized and there is chaos in the CGM world -
enough to endanger its success as a standard.

I spoke to Russ Rew and he agreed that a "straw man" proposal on these
conventions for oceanographers was in order.  I will try to pull one
together in the next few days - using "conventions.info" as a starting point
but going into much greater detail.  My main goal will be to enumerate the
open issues.  The list I generate will be VERY incomplete - I hope we can
pass it around and add to it.  When we have a moderately exhaustive list
then we can begin discussing solutions that encompass our issues.

If you see a problem with this process please fire away!

	cheers - steve