John
I believe users read from netCDF vs grib files both because reading from
grib is not easy and because netCDF provides
metadata that is human readable. Changing variable names to
VAR__.... is not human readable. Users doing a ncdump would be
forced to read through non-standard metadata attributes to see what the variable
was (or refer to grib tables which would be different
for grib1 vs grib2). These grib tables wouldn't necessarily be handy to
most users. Users who have scripts now to get data
would have to change their scripts in a way that isn't straightforward
and also (to me) seems more subject to typos and similar errors.
While I see that for some developers, the change might make things
easier, it would not be for all developers and it certainly would not
for most scientists and researchers using TDS files directly. They don't
have the time or patience to have to look these things up when they just
want to know about the contents of a file quickly, which IS possible now via
TDS.
Cathy Smith
NOAA/ESEL PSD
On 2/27/12 11:17 AM, John Caron wrote:
> To all:
>
> The CDM / netCDF-Java library version 4.3 (and also TDS version 4.3)
> is considering a radical change in the way that GRIB variables are
> named. Instead of nice human readable names like
>
> float Temperature(time=1, lat=361, lon=720);
>
> they are now like
>
> float VAR_0-0-0_L6_I6_Hour_S194(time=1, lat=361, lon=720);
>
> with "human readable names" in the long_name:
>
> :long_name = "Temperature (6_Hour Average) @ Maximum wind level";
>
> The reasons for this change are that the "nice human readable names"
> come from external GRIB tables, that is, the names are not in the
> files themselves. GRIB table parameter names have no requirement to be
> unique nor simple nor unchanging, i.e. they have no requirement to be
> suitable as netCDF variable names. Maintainers of GRIB tables often
> make minor changes to GRIB names, correcting typos or otherwise
> improving the readability of the name. In some cases, the GRIB names
> are completely changed. When the CDM starts to use new versions of
> the tables, the variable names can and do change. Since calls to
> access data use the name of the variable, many things break if the
> name changes.
>
> Any GRIB to netCDF translation software is in the position of either
> hand-maintaining the tables to prevent names from changing (and fixing
> duplicates or unsuitable names), or doing something else. Hand
> maintaining GRIB tables is not a viable option due to resource
> constraints. The something else is to give variables unique names
> based only on the information actually in the file. The NCL package
> has adopted a similar scheme:
>
> http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/NclFormatSupport.shtml#GRIB
>
>
> More background on this problem is here:
>
> http://www.unidata.ucar.edu/staff/caron/papers/GRIBarchivals.pdf
>
> Another aspect of this problem is that errors were found in version
> 4.2 with GRIB tables, with handling GRIB time intervals and ensemble
> data, as well as with the algorithm for generating names when multiple
> variables from the same parameter are in the same file. About 1 in 5
> variable names (in the NCEP IDD data) need to change to fix these
> problems. In reviewing how variable names are created, and how GRIB
> tables are handled, these other problems became clear. Rather than
> fixing the problem piecemeal, we are trying to make one big change
> all at once, then do our best to not let this happen again.
>
> The main impact this will have is probably on:
> 1) scripts or IDV bundles that have a GRIB variable name in them;
> hopefully a one-time change will fix this.
> 2) interactive applications that are built on top of the CDM. For
> GRIB, users will need to see the long_name, not the variable name, to
> know what they want. However, the CDM presents a uniform interface for
> all files, not just GRIB, so the application can't assume that the
> long_name is even present. So the application should present both the
> variable name and the long_name (if it exists) to the user when
> selecting variables.
>
> We think that this change, though painful, is a necessary way forward,
> but we want to get input from users, and especially application
> developers. The latest 4.3 snapshot has these changes, please try it
> out and let us know what you think, and how it will affect you. Post
> your comments to these email lists so the entire discussion can be
> public.
>
> thanks
>
> John, Ethan
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
--
----------------------------------------------
NOAA/ESRL PSD and CIRES CDC
303-497-6263
http://www.esrl.noaa.gov/psd/people/cathy.smith/
Emails about data/webpages may get quicker responses from emailing
esrl.psd.data@xxxxxxxx
