[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: LDM blues

Chris Herbster wrote:
> 
> Anne,
> 
> What do you recommend for a queue size?  What about separate queues for
> different product types - isn't that an alternative structure?
>

No, the ldm doesn't support writing different products to different
queues.

Regarding size, wellll, it all depends, of course, on what you want.  

  "Cheshire," Alice began rather timidly, "would you tell me please,
   which way I ought to go from here?" "That all depends a good deal
   on where you want to get to," said the Cat. 

Sorry, just reminiscing...

Of course, what you request also depends on what your connection can
handle, as you've seen.  It also depends to some degree on your
hardware, but you weren't asking for much even before you cut back on
your requests.  At the bottom if this message I appended some statistics
taken last August for the feeds you were requesting.  From these, you
can get a sense of the volume of the various feeds.  Bear in mind that
the volume does vary and this is just a snapshot.  Looks like a 500Mb
queue would allow you to comfortably keep at least an hour's worth of
data in the queue.

After your queue has reached a steady state, run pqmon to see the age of
the oldest product in the queue.  For you as a leaf node, I recommend
that that age be about 45 to 60 minutes.  If you do start propagating to
Arizona, I would say 60 minutes is a minimum. 

> 
> I was discussing this networking problem with one of our IT support people
> earlier today.  I told him how we were supposed to have a NOAAPort system by
> now, but the University administration didn't give us the $35K that we had in
> our budget for our new degree program hardware needs.  (This was a budget item
> that was previously approved with question.)  I pointed out that this would 
> give
> us the equivalent of about 4 T1 lines worth of data bandwidth.  Since this
> purchase would be a one time shot and the T1's are regularly recurring fees, 
> he
> might find the money in a different spot of his budget to get us off the IDD
> (for most products).
> 

If connectivity is really a problem for you this might be a good
alternative.  There is some maintenence involved.  And, I *think*
[disclaimer: ON] that the SSEC systems are at $15K.

> I've discussed this briefly with different Unidata folks in previous years.  
> We
> would like to have a NOAAPort system that is free of proprietary software - 
> i.e.
> use the Unidata Apps. as our infrastructure.  I assume that this is what you
> guys do, right?
> 

Not really.  For our "production" boxes we use the SSEC ingest software,
which is proprietary.  But, some people here also built a prototype
ingest system that has been running for some time.  I guess that's a
possibility, but it require some expertise, which I, for example, don't
currently have.

> BTW, we already have the 13' dish.  We just need the hardware, software and 
> have
> to re-point the dish to the correct satellite....
> 

I asked Mike Schmidt about this.  He was surprised to hear that your
dish is on 13".  He thought a 2 meter dish was more appropriate.  This
is a bit out of my league.

Why do you want nonproprietary system?  One benefit of a proprietary
system is that you get everything already configured, and you get
support.  Our SSEC ingest systems runs very well with very little
maintenence.  Just playing devil's advocate...

> Thanks again for your help.  Hopefully we'll have some info from the network
> folks next week.
> 
> Have a great weekend!
> 
> Best wishes,
> 
> CH
> 

Good luck with your network people!

Anne

-- 
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                 P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************

Stats gathered over roughly a 24 hour period:
--------------------------------- IDS_DDPLUS ------
num IDS_DDPLUS products = 160508
total IDS_DDPLUS bytes      =     157 Mb
avg IDS_DDPLUS product size =     979 bytes
min IDS_DDPLUS bytes/hour   =    1669 bytes
max IDS_DDPLUS bytes/hour   = 6925873 bytes
--------------------------------- HDS ------
num HDS products = 204943
total HDS bytes      =   2150 Mb
avg HDS product size =  10493 bytes
min HDS bytes/hour   =  41150 bytes
max HDS bytes/hour   =    197 Mb
--------------------------------- MCIDAS ------
num MCIDAS products = 365
total MCIDAS bytes      =     131 Mb
avg MCIDAS product size =  359047 bytes
min MCIDAS bytes/hour   = 2647666 bytes
max MCIDAS bytes/hour   = 7094599 bytes
--------------------------------- NLDN ------
num NLDN products = 244
total NLDN bytes      =      8 Mb
avg NLDN product size =  34312 bytes
min NLDN bytes/hour   =  79548 bytes
max NLDN bytes/hour   = 614152 bytes
--------------------------------- PCWS ------
num PCWS products = 147
total PCWS bytes      =      95 Mb
avg PCWS product size =  647261 bytes
min PCWS bytes/hour   =  1602616 bytes
max PCWS bytes/hour   = 5779704 bytes