[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Status

On Wed, 5 Jan 2000, Ekaterina Radeva wrote:

> Robb,
> 
> Happy New year and all the best in the new millenium !

 Ekaterina,

Thanks for the greeting.  I wish you the best in the millenium also.

The size and product ID's look good. So it's
looks like we need to set up a test time, etc.  The rest of this week and
next week are all tied up with preparation and attending the AMS
conference in LA.  So  in two weeks we can start the process. I'll
e-mail you when I get back or if I somehow forget, remind me and we can
get started.

Robb...

> 
> I think we are ready to test our ldm configuration.
> 
> By means of the pqinsert utility, I put a subset of the products we would
> like to distribute via the Unidata IDD system in our local queue.
> 
> The ldm host machime here is ftp.cmc.ec.gc.ca and the queue path is
> /data/dns2/ldm/ldm.pq (different from the default path). I included the
> ALLOW statement for shemp.unidata.ucar.edu in etc/ldmd.conf. 
> 
> I put in the queue 80 products (GRIB files) with names of the type:
> CMC_$var_$level_2000010500_$phour, where 
> 
> $var is the field constituting the file (tt=temperature; es=dew-point
> depression; gz=geopotential height; p0=sfc pressure; pn=sea-level
> pressure; uv=u- and v- wind components; ww=vertival velocity;
> pr=precipitation accumulation; rt=precipitation rate; nt=cloud cover)
> 
> $level is 1000, 850, 700, 500, 400, 250, 150, 100 hPa + sfc
> 
> 2000010500 is the initial time of our regional model forecast,
> 
> and $phour is the forecast hour, in this case 0 or 6. When the
> transmission strts for real, we plan to send forecasts up to 48 hour, at
> every 6 hours. 
> 
> The size of the products/files is 19K, except for the wind where both
> components are included in the same file and the size doubles.
> 
> Once you receive the GRIB files, I would like you to try decoding them
> with the Unidata GRIB decoder. I guess that this is the decoder of choice
> for the Unidata university users, so we have to make sure that it is
> compatible with our GRIB. I could test the decoder here, but it will
> produce the netCDF files, which we are not familiar with. 
> 
> This is all for now. Let me know how it goes. I'll keep my fingers
> crossed.
> 
> With best regards,
> Ekaterina
> 
> On Tue, 28 Dec 1999, Robb Kambic wrote:
> 
> > On Tue, 21 Dec 1999, Ekaterina Radeva wrote:
> > 
> > > Hello Linda and Robb,
> > > 
> > > In response to your e-mail, here is where we stand: the LDM software was
> > > installed on one of our external servers which is frequently used as host
> > > for outgoing data. Now we would like to run some tests by sending data to
> > > another IDD server, some of your machines for instance. 
> > 
> > Ekaterina,
> > 
> > 
> > It sounds like you have a handle on getting this operation in gear.  Yes,
> > I think it's a good idea to send data to one of our machines as a test.
> > The machine will be shemp.unidata.ucar.edu.  You will need to put an ALLOW
> > statment in your etc/ldmd.conf file for our machine, ie
> > 
> > allow ANY shemp.unidata.ucar.edu
> > 
> > Also, I will need the name of your machine so I can request the GEM model.
> > > 
> > > Technicalities
> > > 
> > > In general, we would like to send over the IDD system output from our
> > > regional GEM model in grib format. As a starting point, we propose the
> > > following subset of the GEM output to be sent to the UNIDATA users. Feel
> > > free to feedback your suggestions/comments.
> > > 
> > > Fields (49 in total):
> > > 
> > > - geopotential height at 1000, 850, 700, 500, 400, 250, 150, and 100 hPa
> > > - air temperature at the same pressure levels + at the model surface 
> > > - wind components projected on the x- and y- axies of the grid at the
> > >   same pressure levels + at the model surface 
> > > - dew-point temperature at the same pressure levels + at the model
> > >   surface 
> > > - sea-level pressure 
> > > - pressure vertical velocity at 700 hPa 
> > > - surface pressure 
> > > - accumulated precipitation 
> > > - instantaneous rainfall rate 
> > > - total column cloud cover
> > > 
> > > Grid:
> > > 
> > > A polar-stereographic grid over Canada, the U.S. and the adjacent waters.
> > > 135x94 points, horizontal resolution of 60km at 60N.
> > > 
> > > The regional operational model is run twice per day based on 00Z and 12Z
> > > data, to produce 48-hour forecast. We would like to send the the 0-48 hour
> > > forecasts of the above fields from both runs, at a temporal resolution of
> > > 6 hours. This will make for 882 fields/products with a volume of ~ 17Mb
> > > per day. This seems reasonable, having in mind that the model output from
> > > the NMC and ECMWF models you disseminate accounts for 20,000 products and
> > > 75 MB per day. However, you may have restrictions for experimental data.
> > > 
> > > As far as I understood from the documentation, each field is sent as a
> > > separate file/product so that the users can select the fields they need.
> > > Is this so, or can we group several fields as one product ? Say, all
> > > fields for a certain forecast hour ? It seems to me that this will make
> > > for less trafic.
> > > 
> > You are correct 17Mb is not much data for a 24 hour period. I would
> > suggest that you break the GEM model into products that are the most
> > useful to the end user that are about 16k apiece. If its by parameter or
> > time it's up to you.  The only restriction would be not to send the whole
> > GEM model as one product because if there is sending problem then the
> > whole product would have to be resent again.  Also, the product name
> > should reflect the contents and time but don't add the complete directory
> > structure to the product name. 
> > 
> > 
> > > As for GRIB encoding, we use an in-house encoder which is consistent with
> > > the WMO GRIB (edition 1) standards. I will get your GRIB decoder from the
> > > UNIDATA site and check whether it works with our GRIB data. Is your GRIB
> > > decoder decoding into ASCII or some other internal format ? If so, the
> > > decoding test should probably be run at your site, as we may not be
> > > familiar with the internal format.
> > > 
> > Unidata GRIB decoder "gribtonc" will read WMO GRIB (edition 1) standards
> > and the output is a netCDF file. If you have a sample GEM GRID file, I
> > could test it for you or you can install the Unidata decoders package,
> > that requires installing netCDF, udunits, and the decoders packages. All
> > the packages are available by ftp.unidata.ucar.edu or their respective web
> > pages.  Another idea, I could make available a gribdump routine in binary
> > format for you platform. I don't know the platform type.
> > 
> > 
> > 
> > > As a simple test, I guess I can use the pqinsert/pqsend utilities to send
> > > you a few files using the LDM software ? I can use the -f option to
> > > specify an EXP feedtype. I will also need the address of the remote
> > > server I am to send the data to. 
> > > 
> > For the test, you could use the pqinsert utility to put the product in the
> > LDM queue and the request from our machine will take care of the sending
> > part. Yes, use the -f flag with -f EXP.
> > 
> > 
> > > I'll appreciate any advice about how to use the LDM, what test we can make
> > > and how to finish the set-up. Thanks in advance,
> > > 
> > If you need any more info or if I confused you on some of the brief
> > explanations, let me know.
> > 
> > Robb...
> > 
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================