Hi all,
[Sorry for cross-posting. If you access GRIB data from the TDS (THREDDS
Data Server) on motherlode.ucar.edu, please read on.]
We recently promoted netCDF-Java 4.3 and TDS 4.3 to be our stable
releases. We are currently working on plans to upgrade the Unidata TDS
[1] to run TDS 4.3. As discussed in earlier emails, this will cause
the names of some variables in GRIB datasets to change. (For more
details, see note [2] below.) We need help now to test the impact of
these changes on you, rather than waiting until the switch is made on
the production server.
This change to the names of variables will require some work for users
and application developers if their applications or scripts access
GRIB data through the Unidata TDS especially if that access is done in
a scripted manner (e.g., with variable names hard-coded in a script or
persisted in some other way).
To assist in this effort, the netCDF-Java library contains a mapping
between old variable names and new variable names for all the NCEP
GRIB model data available from the Unidata TDS. However, there is not
always an unambiguous mapping from the old variable name to the new
variable name. So we strongly recommend that you carefully review the
new variables and select which ones you actually want to use. Examine
the metadata carefully, including the time coordinate, especially if
they are time intervals (e.g., an average or accumulation). These time
interval variables are likely to be incorrect in 4.2. Also, please
note that the variable name mapping may not be accurate for non-NCEP
GRIB data. The variable name mapping information is available in an XML
file in the netCDF-Java library. The netCDF-Java library also provided
an API for programatic access to the variable name mapping information.
(See links in [3] for documentation on the variable name mapping XML
file and API.)
The Unidata Test TDS [4] is currently running the latest TDS
4.3. Please test your scripts and applications against that server.
The THREDDS and IDV teams are working closely to make sure the IDV can
deal with this transition in a relatively seamless way (with an update
to a new version of the IDV and some minimal user interaction). When
opening IDV bundles that reference affected variable names, the IDV
will use the netCDF-Java variable name mapping API in an attempt to
find the new variable names. If that fails, the IDV will request input
from the user. As described above, we recommend that IDV users
manually review the new variables to make sure these are the ones you
actually want to use.
We plan to upgrade the Unidata TDS to 4.3 shortly after IDV 4.0 is
released. We hope this will occur in the next few weeks but will
provide more information on the timeline as we move forward. In the
mean time, please test your scripts and tools against the Unidata Test
TDS [4].
Thanks,
Unidata THREDDS Team (Ethan, John, Marcos, Lansing, and Sean)
Unidata IDV Team (Yuan, Julien, and Sean)
=====
[1] Note on the URL for the Unidata TDS: We are deprecating
http://motherlode.ucar.edu:8080/thredds/ in favor of
http://thredds.ucar.edu/thredds/. The motherlode.ucar.edu:8080 URL
will continue to function for now; we will stop supporting it at some
point in the future.
[2] CDM/TDS 4.2 uses an incorrect mapping from GRIB records to netCDF
variables. CDM/TDS 4.3 has fixed the GRIB to netCDF mapping to more
accurately capture the scientific meaning of the variables. This fix
resulted in changes to some of the resulting netCDF variable names,
descriptions, and metadata (and ended up requiring a fairly complete
rewrite of the GRIB reading code). Unfortunately, the variable name
changes will require some work for users and developers if their
applications or scripts access GRIB data through a TDS especially if
that access is done in a scripted manner (e.g., with variable names
hard-coded in a script or persisted in some other way).
For more details on the problem of mapping GRIB records to netCDF
variables, please see:
http://www.unidata.ucar.edu/staff/caron/presentations/GribNaming.pptx
For details on how GRIB variables are named in CDM/TDS 4.3, please see
the "Mapping a GRIB Collection into Multidimensional Variables"
section of
http://www.unidata.ucar.edu/software/netcdf-java/formats/GribFiles.html
[3]
http://www.unidata.ucar.edu/projects/THREDDS/tech/tds4.3/UpgradingTo4.3.html
[4] http://thredds-test.ucar.edu/thredds/
