[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: NetCDF/HDF5 Proposal Approved (fwd)



Authur,

My name is Robb Kambic, I'm in charge of the grib decoders at Unidata.
Thanks for the url about your work, I'll look at it.  I'm also working on
another project called Thematic Realtime Environmental Data Distributed Services
(THREDDS).  The idea is use/add extra metadata to the grib2 products
to be harvested by data crawlers to make metadata catalogs containing this
information and urls to the actual datasets. So I'm interested in the
metadata aspects of grib2. Maybe we can collaborate on the metadata in the
grib2 files.  At this time we are still working on the grib/NetCDF
conventions for our grib1 files, even though they been in use for years.

For instance variables might look like:

        float  T_fhg(record,fhg,lat,lon) ;
               T_fhg:long_name = "Temperature at fixed height above ground" ;
               T_fhg:parm_number = 11 ;
               T_fhg:level = 105 ;
               T_fhg:units = "degK" ;
               T_fhg:_FillValue = -9999.f ;
               T_fhg:navigation = "nav" ;

Since the parmeter/level numbers are included, an external table could be
used to match the parameter name or the long_name. Also, I'm attaching a
NetCDF template file (cdl) for the mrf model for the other conventions.
The new grib1 conventions should be rolled into the grib2 files, hopefully
with some new metadata. I'll keep you informed of our progress.

Robb...


On Mon, 17 Mar 2003, Russ Rew wrote:

> Robb,
>
> Here's more information to file away for the GRIB2 conversion some day ...
>
> --Russ
>
> ------- Forwarded Message
>
> Date:    Mon, 17 Mar 2003 13:38:21 -0500
> From:    "Arthur.Taylor" <address@hidden>
> To:      Bill Cassanova <address@hidden>
> cc:      Russ Rew <address@hidden>, David Ruth <address@hidden>,
>        Timothy Boyer <address@hidden>
> Subject: Re: NetCDF/HDF5 Proposal Approved
>
> This is a multi-part message in MIME format.
> --------------5AFE1FB09110D4800D24A980
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
>
> Hi all,
>
> I'm somewhat interested, but I am not very familiar with NetCDF (so please 
> forgive me if my
> terminology is off).  My problem is that NetCDF seems a little too free form, 
> so I wasn't sure what
> the template should contain.  For example, how much of the GRIB2 meta data to 
> preserve.  I started
> coming up with at template a while ago, but I had to put it on hold for a bit 
> while I worked on a
> different project.
>
> My rough draft template is attached.  My original inclination was to preserve 
> as much of the GRIB2
> meta data as I could, but some of the feedback I got was to reduce it.  For 
> example, one probably
> doesn't need "GRIB2 Master Tables Version" in the NetCDF file.
>
> -----
>
> Bill, since you are more familiar with how one might use a GRIB2 message in 
> NetCDF, could you glance
> over it?  I was under the impression that you were using my "degrib" program 
> to convert from GRIB2 to
> a NxM floating point file, and then had written something to convert the 
> floating point file to
> NetCDF.  If that is the case, what NetCDF template did you use, and should I 
> base mine on it?
>
> -----
>
> Russ, you may want your decoders developer to check out the following site 
> when dealing with GRIB2...
> www.nws.noaa.gov/mdl/NDFD_GRIB2Decoder  I think it may be helpful for him, 
> since it should contain
> links on the "howto.htm" page to a GRIB2 decoder library, and provide 
> examples as to how to call the
> library.
>
> Regards,
> Arthur
>
> Bill Cassanova wrote:
>
> > There might be a guy at NOAA that would be interested in helping with this 
> > as he
> > is in the process of writing several Grib2
> > utilities...I don't know but this might be a good oportunity for the 2 
> > groups to
> > work together.
> >
> > His name is Arthur Taylor.
> > <address@hidden>
> >
> >
> >                       Russ Rew
> >                       <address@hidden         To:      "Bill Cassanova" 
> > <address@hidden>
> >                       ar.edu>                  cc:      address@hidden
> >                                                Subject: Re: NetCDF/HDF5 
> > Proposal Approved
> >                       03/17/03 11:16
> >                       AM
> >
> >
> >
> > Hi Bill,
> >
> > > Any thoughts on a Grib2 to netcdf converter?
> >
> > No, sorry, it's not part of the NASA-funded project.  We will have to
> > update our GRIB to netCDF converter, gribtonc, eventually to handle
> > GRIB2.  I'll CC: our decoders developer, Robb Kambic, who maintains
> > gribtonc, in case he has anything to add.
> >
> > --Russ
>
> --------------5AFE1FB09110D4800D24A980
> Content-Type: text/plain; charset=us-ascii;
>  name="netcdf.txt"
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline;
>  filename="netcdf.txt"
>
> GRIB2 Message Based NetCdf:
>
> /****************************************************************************/
>   Notes:
> char               8-bit characters intended for representing text.
> byte               8-bit signed or unsigned integers (see discussion below).
> short              16-bit signed integers.
> int                32-bit signed integers.
> float or real      32-bit IEEE floating-point.
> double             64-bit IEEE floating-point.
>
>     May want scanY to be positive...
>     Missing: WxTable, and sect4_Interval type:
>
> typedef struct {
>    char **data;              /* Array of text strings (aka "ugly strings) */
>    long int dataLen;         /* number of text strings in data. */
> } sect2_WxType;
>
> /****************************************************************************/
>
> dimensions:
>    Nx = 1073;
>    Ny = 689;
>    DimInterval = 256;
> variables:
>    short S0_prodType;
>       S0_prodType:long_name = "Discipline";
>    int S1_centre;
>       S1_centre:long_name = "Originating Centre";
>    int S1_subcentre;
>       S1_subcentre:long_name = "Sub Centre";
>    short S1_mstrVersion;
>       S1_mstrVersion:long_name = "GRIB2 Master Tables Version";
>    short S1_lclVersion;
>       S1_lclVersion:long_name = "GRIB2 Local Tables Version";
>    short S1_sigTime;
>       S1_sigTime:long_name = "Significance of Reference Time";
>    double S1_refTime;
>       S1_refTime:long_name = "Reference Time";
>       S1_refTime:units = "seconds since 1-1-1970";
>    short S1_operStatus;
>       S1_operStatus:long_name = "Operational Status";
>    short S1_dataType;
>       S1_dataType:long_name = "Type of Data";
>    short S3_projType;
>       S3_projType:long_name = "Map Projection Type";
>    short S3_shapeEarth;
>       S3_shapeEarth:long_name = "Shape of the Earth";
> /*--------*/
> /* Depends on S3_shapeEarth... following for sphere. */
>    double S3_radEarth;
>       S3_radEarth:long_name = "Radius of the Earth";
>       S3_radEarth:units = "km";
> /*--------*/
> /* Depends on S3_shapeEarth... following for elipsoid. */
>    double S3_majEarth;
>       S3_majEarth:long_name = "semi major axis of Earth";
>       S3_majEarth:units = "km";
>    double S3_minEarth;
>       S3_minEarth:long_name = "semi major axis of Earth";
>       S3_minEarth:units = "km";
> /*--------*/
> /* Following should match the dimmensions Nx, Ny */
>    int S3_NumX;
>       S3_NumX:long_name = "Number of points on a parallel";
>    int S3_NumY;
>       S3_NumY:long_name = "Number of points on a meridian";
> /*--------*/
>    double S3_Lat1;
>       S3_Lat1:long_name = "Latitude of first grid cell";
>       S3_Lat1:units = "degrees north";
>    double S3_Lon1;
>       S3_Lon1:long_name = "Longitude of first grid cell";
>       S3_Lon1:units = "degrees east";
> /*--------*/
> /* Depends on S3_projType... following for lat/lon grid. */
>    double S3_Lat2;
>       S3_Lat2:long_name = "Latitude of last grid cell";
>       S3_Lat2:units = "degrees north";
>    double S3_Lon2;
>       S3_Lon2:long_name = "Longitude of last grid cell";
>       S3_Lon2:units = "degrees east";
> /*--------*/
>    double S3_Di;
>       S3_Di:long_name = "Mesh size in x direction";
>       S3_Di:units = "mm";
>    double S3_Dj;
>       S3_Dj:long_name = "Mesh size in y direction";
>       S3_Dj:units = "mm";
>    double S3_meshLat;
>       S3_meshLat:long_name = "Latitude where Mesh size is correct";
>       S3_meshLat:units = "degrees north";
>    double S3_orientLon;
>       S3_orientLon:long_name = "Longitude where north is up";
>       S3_orientLon:units = "degrees east";
>    double S3_scaleLat1;
>       S3_scaleLat1:long_name = "Tangent Latitude, or a latitude which should 
> have the same scale as scaleLat2";
>       S3_scaleLat1:units = "degrees north";
>    double S3_scaleLat2;
>       S3_scaleLat2:long_name = "Tangent Latitude, or a latitude which should 
> have the same scale as scaleLat1";
>       S3_scaleLat2:units = "degrees north";
>    int S4_templat;
>       S4_templat:long_name = "Product Definition Template Number";
>    short S4_cat;
>       S4_cat:long_name = "Category Description";
>    short S4_subcat;
>       S4_subcat:long_name = "Category Sub-Description";
>    short S4_genProcess;
>       S4_genProcess:long_name = "Generation Process";
>    short S4_bgGenID;
>       S4_bgGenID:long_name = "Background Generating Process ID";
>    short S4_genID;
>       S4_genID:long_name = "Forecast Generating Process ID";
> /*--------*/
> /* If f_validCutOff :... */
>    int S4_cutOff;
>       S4_cutOff:long_name = "Data Cutoff after S1_refTime";
>       S4_cutOff:units = "seconds since S1_refTime";
> /*--------*/
>    double S4_foreTime;
>       S4_foreTime:long_name = "Forecast time";
>       S4_foreTime:units = "seconds since S1_refTime";
>    short S4_fstSurfType;
>       S4_fstSurfType:long_name = "Type of first fixed surface";
>    double S4_fstSurfValue;
>       S4_fstSurfValue:long_name = "Value of first fixed surface";
> /*--------*/
> /* If sndSurfType ! missing:... */
>    short S4_sndSurfType;
>       S4_sndSurfType:long_name = "Type of second fixed surface";
>    double S4_sndSurfValue;
>       S4_sndSurfValue:long_name = "Value of second fixed surface";
> /*--------*/
>    double S4_validTime;
>       S4_validTime:long_name = "Valid Time";
>       S4_validTime:units = "seconds since 1-1-1970";
> /*--------*/
> /* If template 4.1... */
>    short S4_typeEnsemble;
>       S4_typeEnsemble:long_name = "Type of Ensemble Forecast";
>    short S4_perturbNum;
>       S4_perturbNum:long_name = "Perturbation number";
> /*--------*/
> /* If template 4.1, 4.2 ... */
>    short S4_numberFcsts;
>       S4_numberFcsts:long_name = "Number of Forecasts in Ensemble";
> /*--------*/
> /* If template 4.2 ... */
>    short S4_derivedFcst;
>       S4_derivedFcst:long_name = "Derived Forecast";
> /*--------*/
> /* If template 4.8, 4.9 ... */
>    short S4_numInterval;
>       S4_numInterval:long_name = "Number of time intervals";
>    int S4_numMissing;
>       S4_numMissing:long_name = "Number of missing values";
> /* If numInterval != 0 */
>    short S4_processID(DimInterval);
>       S4_processID:long_name = "Statistical process method used";
>    short S4_incrType(DimInterval);
>       S4_incrType:long_name = "Type of time increment between intervals";
>    int S4_timeIncr(DimInterval);
>       S4_timeIncr:long_name = "Time increment between intervals";
>    short S4_timeIncrUnit(DimInterval);
>       S4_timeIncr:long_name = "Unit of time incremnt";
>    int S4_lenTime(DimInterval);
>       S4_lenTime:long_name = "Range or Length of time interval";
>    short S4_lenTimeUnit(DimInterval);
>       S4_lenTimeUnit:long_name = "Unit of Range of time interval";
>    short S4_foreProbNum;
>       S4_foreProbNum:long_name = "Forecast probability number";
>    short S4_numForeProbs;
>       S4_numForeProbs:long_name = "Total number of Forecast Probabilities";
>    short S4_probType;
>       S4_probType:long_name = "Type of Probability Range";
>    double S4_lowerLimit;
>       S4_lowerLimit:long_name = "Lower Limit of Probability Field";
>    double S4_upperLimit;
>       S4_lowerLimit:long_name = "Upper Limit of Probability Field";
> /*--------*/
>    short S5_missManage;
>       S5_missManage:long_name = "Missing Value Management";
>    double S5_missPrim;
>       S5_missPrim:long_name = "Primary Missing Value";
>    double S5_missSec;
>       S5_missSec:long_name = "Secondary Missing Value";
>    double S5_min;
>       S5_min:long_name = "Minimum value detected in data";
>    double S5_max;
>       S5_max:long_name = "Maximum value detected in data";
>    double Data(Nx,Ny);
>       Data:long_name = "<*element: 'MaxT', 'MinT', 'Wx', etc >";
>       Data:units = "<*unitName: 'Knots', 'm/s', etc>";
>       Data:scanX = "Scan i/x direction positive";
>       Data:scanY = "Scan j/y direction negative";
>       Data:scanConsecutive = "Scan consecutive points in i/x direction";
>       Data:scanAdjacent = "Scan adjacent rows in same direction";
>       Data:windVect = "u/v vectors relative to east/north";
> data:
>
>
> --------------5AFE1FB09110D4800D24A980--
>
> ------- End of Forwarded Message
>
>

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================
netcdf mrf-a{ 


dimensions:
        record = UNLIMITED ;   // (reference time, forecast time)
        lat = 73 ;             // latitude
        lon = 73 ;             // longitude
        level = 1 ;            // isobaric levels
        fhg = 1 ;              // fixed height above ground
        datetime_len = 21 ;    // string length for datetime strings
        valtime_offset = 8 ;   // number of offset times
        nmodels = 3 ;          // number of models
        ngrids = 4 ;           // number of grids
        nav = 1 ;              // for navigation
        nav_len = 100 ;        // max string length for navigation strings

variables:

        double reftime(record) ;        // reference time of the model
               reftime:long_name = "reference time" ;
               reftime:units = "hours since 1992-1-1" ;

        double valtime(record) ;        // forecast time ("valid" time)
               valtime:long_name = "valid time" ;
               valtime:units = "hours since 1992-1-1" ;

        :record = "reftime, valtime" ;  // "dimension attribute" -- means
                                        // (reftime, valtime) uniquely
                                        // determine record

        char   datetime(record, datetime_len) ; // derived from reftime
               datetime:long_name = "reference date and time" ;
               // units YYYY-MM-DD hh:mm:ssZ  (ISO 8601)

        double valtime_offset(valtime_offset) ; // valtime - reftime
               valtime_offset:long_name = "hours from reference time" ;
               valtime_offset:units = "hours" ;

        float  level(level) ;  // isobaric levels
               level:long_name = "level" ;
               level:units = "hectopascals" ;

        float  fhg(fhg) ;
               fhg:long_name = "fixed height above ground" ;
               fhg:units = "meters" ;


        // The following lat and lon coordinate variables are redundant,
        // since the navigation variables provide the necessary information.
        // The extra information is included here for human readability.

        float  lat(lat) ;
               lat:long_name = "latitude" ;
               lat:units = "degrees_north" ;

        float  lon(lon) ;
               lon:long_name = "longitude" ;
               lon:units = "degrees_east" ;

        long   model_id(nmodels) ;
               model_id:long_name = "generating process ID number" ;

        // navigation variables all use nav dimension

        char   nav_model(nav, nav_len) ;        // navigation parameterization
               nav_model:long_name = "navigation model name" ;

        int    grid_type_code(nav) ;
               grid_type_code:long_name = "GRIB-1 GDS data representation type" 
;

        char   grid_type(nav, nav_len) ;
               grid_type:long_name = "GRIB-1 grid type" ;

        char   grid_name(nav, nav_len) ;
               grid_name:long_name = "grid name" ;

        int    grid_center(nav) ;
               grid_center:long_name = "GRIB-1 originating center ID" ;

        int    grid_number(nav, ngrids) ;
               grid_number:long_name = "GRIB-1 catalogued grid numbers" ;
               grid_number:_FillValue = -9999 ;

        char   i_dim(nav, nav_len) ;
               i_dim:long_name = "longitude dimension name" ;

        char   j_dim(nav, nav_len) ;
               j_dim:long_name = "latitude dimension name" ;

        int    Ni(nav) ;
               Ni:long_name = "number of points along a latitude circle" ;

        int    Nj(nav) ;
               Nj:long_name = "number of points along a longitude circle" ;

        float  La1(nav) ;
               La1:long_name = "latitude of first grid point" ;
               La1:units = "degrees_north" ;

        float  Lo1(nav) ;
               Lo1:long_name = "longitude of first grid point" ;
               Lo1:units = "degrees_east" ;

        float  La2(nav) ;
               La2:long_name = "latitude of last grid point" ;
               La2:units = "degrees_north" ;

        float  Lo2(nav) ;
               Lo2:long_name = "longitude of last grid point" ;
               Lo2:units = "degrees_east" ;

        float  Di(nav) ;
               Di:long_name = "longitudinal direction increment" ;
               Di:units = "degrees" ;

        float  Dj(nav) ;
               Dj:long_name = "latitudinal direction increment" ;
               Dj:units = "degrees" ;

        byte   ResCompFlag(nav) ;
               ResCompFlag:long_name = "resolution and component flags" ;

        // end of navigation variables

        float  Z(record, level, lat, lon) ;
               Z:long_name = "Geopotential height at isobaric levels" ;
               Z:units = "gp m" ;
               Z:_FillValue = -9999.f ;
               Z:navigation = "nav" ;  // georeference info

        float  T_fhg(record,fhg,lat,lon) ;
               T_fhg:long_name = "Temperature at fixed height above ground" ;
               T_fhg:units = "degK" ;
               T_fhg:_FillValue = -9999.f ;
               T_fhg:navigation = "nav" ;


// global attributes
               :history = "2003-03-12 10:56:20 - created by gribtocdl 1.4 - 
12.12.2002" ; 
               :title = "mrf.wmo" ;
               :Conventions = "NUWG" ;
               :version = 0.0 ;

data:

 level = 1000 ;
 fhg = 2.0 ;
 model_id = 78, 94, 80 ;
 valtime_offset = 84, 96, 108, 120, 132, 144, 192, 240 ;


 // Navigation
 nav_model = "GRIB1" ;
 grid_type_code = 0 ;
 grid_type = "Latitude/Longitude" ;
 grid_name = "Global 5.0 x 2.5 degree grid" ;
 grid_center = 7 ;  // NCEP
 grid_number = 21, 24, 22, 23 ;
 i_dim = "lon" ;
 j_dim = "lat" ;
 Ni = 73 ;
 Nj = 73 ;
 La1 = -90.000000 ;
 Lo1 = -180.000000 ;
 La2 = 90.000000 ;
 Lo2 = 180.000000 ;
 Di = 5.000000 ;
 Dj = 2.500000 ;
 ResCompFlag = 136 ;

 lon = -180,-175,-170,-165,-160,-155,-150,-145,-140,-135,-130,-125,
       -120,-115,-110,-105,-100, -95, -90, -85, -80, -75, -70, -65,
        -60, -55, -50, -45, -40, -35, -30, -25, -20, -15, -10,  -5,
          0,   5,  10,  15,  20,  25,  30,  35,  40,  45,  50,  55,
         60,  65,  70,  75,  80,  85,  90,  95, 100, 105, 110, 115,
        120, 125, 130, 135, 140, 145, 150, 155, 160, 165, 170, 175, 180 ;

 lat = -90.0,-87.5,-85.0,-82.5,-80.0,-77.5,-75.0,-72.5,
       -70.0,-67.5,-65.0,-62.5,-60.0,-57.5,-55.0,-52.5,
       -50.0,-47.5,-45.0,-42.5,-40.0,-37.5,-35.0,-32.5,
       -30.0,-27.5,-25.0,-22.5,-20.0,-17.5,-15.0,-12.5,
       -10.0, -7.5, -5.0, -2.5,  0.0,  2.5,  5.0,  7.5,
        10.0, 12.5, 15.0, 17.5, 20.0, 22.5, 25.0, 27.5,
        30.0, 32.5, 35.0, 37.5, 40.0, 42.5, 45.0, 47.5,
        50.0, 52.5, 55.0, 57.5, 60.0, 62.5, 65.0, 67.5,
        70.0, 72.5, 75.0, 77.5, 80.0, 82.5, 85.0, 87.5, 90.0 ;

}