[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Fwd: NOAAport ingester] (fwd)




===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================

---------- Forwarded message ----------
Date: Tue, 07 Nov 2000 15:09:44 -0700
From: Russ Rew <address@hidden>
To: address@hidden
Subject: Re: [Fwd: NOAAport ingester] (fwd)

Robb,

Here's something I propose to circulate more widely here, to Chiz,
Anne, Jeff, and maybe Ben(?), if you think it's OK.  The third option
discussed in this is something Chiz and I came up with when discussing
whether the feedtype could be assigned by pqing.  Let me know if this
is OK with you to use as a basis for discussion, or if you like, you
could just send it out to the others with whatever changes you want
...

--Russ



Hi,

Robb sent me this draft reply to Jerrold Robaidek <address@hidden>
for comments, but I thought it deserved somewhat wider circulation
because it's important that we convince SSEC of the importance of
tagging radar products with the NNEXRAD feed type when they start
appearing unencrypted in January 2001.

I think we need to discuss this and decide on options for what we will
do if SSEC doesn't go along with creating a third feed for radar data.

I've appended Robb's draft reply with a few minor changes that I made,
for discussion.  Please let us know what you think.

--Russ

To      : Jerrold Robaidek <address@hidden>
Cc      : Russ Dengel <address@hidden>,
          Denise Laitsch <address@hidden>,
          Dave Santek <address@hidden>,
          Dee Wade <address@hidden>  


On Mon, 6 Nov 2000,
Jerrold
Robaidek wrote:

> 
> 
> Hi Robb,
> 
> (I think you are confusing Russ Dengel and I, although we should both
> receive mail regarding this.)
> 
> I have to talk with a few people here about this.  One question ... is
> the reason for changing to the NNEXRAD feedtype to make things easier on
> the IDD community?  

Hiya,


Yes, we believe the additional NEXRAD products added to the current
NWSTG channel will cause problems for many of our sites. With the
current SSEC ingest card configuration, the HDS feed (the current
binary stream) will be expanded to include these radar products.
There will be many radar products (currently over 7000 products/hour
and eventually over 20000 products/hour) and the data volume will at
least double (an additional 80 Mbytes/hour of compressed radar
products and eventually over 200 Mbytes/hour).  The additional
products will cause network congestion for many sites with low
bandwidth and overburden their machines.

As a possible solution, we propose using a separate feed type for the
radar products so that they could be more easily managed. If the SDI
ingest software could be modified to separate out the radar products
into a NNEXRAD feed type, then it could be used in the IDD for routing
or selection.  The question is how practical is it to modify the SSEC
ingest system to create 3 streams (text, binary, and radar) instead of
the current 2 streams (text and binary)?  

As you know, each NOAAPORT product is tagged by a byte in the NOAAPORT
header to belong to one of the "product-specific categories" of image,
graphic, text, grid, point, binary, and other.  For this purpose, we
have reserved the feedtypes NIMAGE, NGRAPH, NTEXT, NGRID, NPOINT,
NOTHER, but we also reserved NNEXRAD for NOAAPORT NEXRAD products,
even though these are currently being classified as "text" (NTEXT).
The compressed products are binary, and there is a compression flag
set for them in the NOAAPORT header; otherwise the compressed products
have exactly the same WMO header as the uncompressed products, so
can't be distinguished from them without the NOAAPORT header
information.

Another solution would be to insert a NOAAport card developed by UPC into
your NOAAport machine that would have the capability to separate out the
NWSTG channel into designated NOAAPORT feedtypes.

> Shouldn't a proper formulation of your request in
> ldmd.conf work?  I think that is how it is being handled now?  

We don't see how it's possible to get another feed type with a change
to the ldmd.conf configuration file, but it would probably be possible
to modify the pqing ingester program to read the WMO header and if it
matches "SDUS5" to assign it the feed type "NNEXRAD".  This is a third
option which we are currently examining to see how easy it would be to
implement.

We need to come up with some way to create a separate radar data
stream.  We would like to continue to discuss the most practical and
desirable way to do this, whether it involves modifying your SDI
software, using our NOAAport card and associated software, or changing
the pqing software.  We hope we can come to an agreement about the
best way to proceed soon, because the radar products will become
available on Jan 1, 2001.

Thanks,
Robb...



 > 
> Thanks for any additional information.
> 
> 
> Jerry
> 
> 
> 
> Russ Dengel wrote:
> > 
> >    Here's Robs letter.
> > 
> > -------- Original Message --------
> > Subject: NOAAport injestor
> > Date: Fri, 3 Nov 2000 14:39:32 -0700 (MST)
> > From: Robb Kambic <address@hidden>
> > Organization: UCAR/Unidata
> > To: address@hidden
> > 
> > Russ,
> > 
> > I meant to talk to you at the workshop about the SSEC NP  ngestor (SDI)
> > software.  As you know the NEXRAD products will be decrpyted on Jan 1,
> > 2001 and UPC would like to send the products out under it's own feedtype
> > NNEXRAD. We believe many sites on the IDD will have congested networks
> > and
> > over burdened machines if the nexrad data is added to the current HDS
> > stream.  So the big question, is it possible to separate out the nexrad
> > products into it's own spool file.  If so then another pqing could read
> > the spool file using the NNEXRAD feedtype. There is another solution, if
> > it's possible to insert the UPC NOAAport card into sunshine?  Our card
> > is
> > developed to read all 4 channels with the proper NOAAport feetypes for
> > the
> > IDD. Of course we would purchase the card and configure it for a pc box.
> > I
> > believe it could go into sunshine box without a problem.
> > 
> > Here's a summary of the IDD feedtypes:
> > 
> > http://www.unidata.ucar.edu/packages/ldm/feedtypes/
> > 
> > I figure there are many pros and cons to this problem, but I wanted to
> > start the communication soon because the holidays in the next month.
> > 
> > Robb...
> > 
> > ===============================================================================
> > Robb Kambic                                Unidata Program Center
> > Software Engineer III                      Univ. Corp for Atmospheric 
> > Research
> > address@hidden                   WWW: http://www.unidata.ucar.edu/
> > ===============================================================================
> 
> -- 
> Jerrold Robaidek                        Email:  address@hidden
> SSEC Data Center                        Phone: (608) 262-6025
> University of Wisconsin                 Fax: (608) 263-6738
> Madison, Wisconsin
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================




------- End of Forwarded Message