Re: Proposed netCDF convention

Steve Hankin (hankin@rogue.pmel.noaa.gov)
Mon, 16 Jun 1997 09:04:45 -0700 

Hi Jonathan,

I have had time only to scan your document but it is clear that some very
careful and thoughtful work has gone into it. Thanks to you and your
colleagues. I was involved in the writing of the COARDS profile (and ANSI
graphics standards) -- I know how much work goes into a document such as yours.

I think we should strive to fold the COARDS document and your work together
(as well as several other issues) to produce COARDS V.2. This would avoid some
of the issues of proliferating standards that Brian Doty raised. Your document
is the correct starting point, as it enumerates exactly the potential V.1/V.2
differences. I believe that you'll find most of the COARDS developers in
agreement that this is a worthwhile goal.

Brian's idea of passing the standard through a larger body such as WMO is a
good one, but for that to succeed requires a much larger committment of time
and energy. I suspect such an effort would need to be sponsored by individuals
who have influence in WMO. (Brian, do you have a suggestion along these
lines?) The COARDS profile has been visibly valuable for standardizing gridded
data interchange ... despite its (obvious) imperfections. It is worth extenting
and improving it.

	- steve

======================================================================

On Jun 13,  9:22am, Jonathan Gregory wrote:
> Subject: Proposed netCDF convention
> Dear Colleague,
>
> We append to this email a proposed convention for the use of netCDF to store
> climate data, especially GCM output. The copy attached here is in HTML, and
can
> be found on the web at http://www-pcmdi.llnl.gov/drach/netCDF.html. It is
also
> available as compressed PostScript in the file netCDF.ps.Z in the same
> directory, and in the file /pub/cr/jmgregory_netcdf.psz through anonymous ftp
> from email.meto.gov.uk. "We" are Jonathan Gregory, Bob Drach and Simon Tett,
> all at the Hadley Centre. Bob is on a year's secondment from PCMDI.
>
> Background information about netCDF: NetCDF (Network Common Data Form) is a
> self-describing machine-independent file format developed by Unidata, which
is
> a programme sponsored by the US National Science Foundation. NetCDF is also a
> freely available software library designed to read and write netCDF files.  A
> substantial number of public domain and commercial data analysis systems can
> accept netCDF files as input. We are interested in netCDF because it offers a
> very simple and flexible set of structures which seem eminently suitable for
> recording climate data and metadata. As netCDF is intended for general
> purposes, conventions on how to use it need to be agreed for specific
> applications such as this. Unidata anticipate this need, and provide a
central
> registry for conventions.
>
> Our principal interest in proposing this convention is to facilitate the
> exchange of data among climate centres. NetCDF offers an appropriate format
for
> this purpose, and these conventions aim to standardise the representation of
> metadata sufficiently that data from different sources can be easily
> compared. We recognise that there are limits to what a standard can
practically
> cover; we restrict ourselves to issues which we believe to be of common and
> frequent concern in the design of climate metadata. Our convention is mostly
> compatible with the existing COARDS convention, but we have extended the
scope
> and detail.
>
> We are aware that some climate centres are already using netCDF as their
> archive format. We would be very interested to have comments from them, for
> instance on how our suggestions differ from what they do, and on what lessons
> they have learned from experience. We would welcome feedback from anyone on
> these conventions, such as on what has been omitted, what could be improved,
> and how we should carry this proposal forward. The exercise will only be
useful
> if it has the support of a number of climate centres, of course. One
> application which will adopt this standard is the LATS software distributed
by
> the Program for Climate Model Diagnosis and Intercomparison (PCMDI), sponsor
of
> AMIP II. LATS will have an option to generate netCDF files which conform to
> this standard.
>
> Our second interest in developing this standard is its relation to a logical
> model of the data and metadata. Describing how the data should be stored in
> netCDF inevitably involves considering how it is organised logically. We hope
> to be make a proposal for a language-independent data model, which could be
> implemented in various programming languages as a method of handling data
> either in memory or in files. If this were done, it would offer a way of
making
> analysis programs more easily portable.
>
> Please send any comments you may have on the proposed standard or any of the
> above to any of us. Feel free to circulate it further if you know others who
> would be interested. Thank you.
>
> Jonathan Gregory   jmgregory@meto.gov.uk
> Bob Drach          drach@llnl.gov
> Simon Tett         sfbtett@meto.gov.uk
>-- End of excerpt from Jonathan Gregory




-- 

		|  NOAA/PMEL               |  ph. (206) 526-6080  
Steve Hankin	|  7600 Sand Point Way NE  |  FAX (206) 526-6744
		|  Seattle, WA 98115-0070  |  hankin@pmel.noaa.gov