[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20040529: CRAFT archival

Inline comments!

Unidata Support wrote:

From: Gerry Creager N5JXS <address@hidden>
Organization: Texas A&M University -- AATLT
Keywords:  200405270011.i4R0BjtK010667 LDM Linux RAID JFS



Gerry,
You may want to look at bigbird. I'm not seeing any Level II data that's not staying bzip'd. 


I switched the pqact action for CRAFT data to filing bzip2 compressed
products directly.  The reason I did this was to see the impact on CPU
use and to see if it would help with the congestion on the RAID.  Also,
the latest GEMPAK distribution 5.2.2p2 (which I loaded on bigbird) can
use the bzip2 compressed images directly.
OK. Then the amount of archive I can maintain is significantly greater. Think we should change scour to, say, 15 days for now?
If that's the case, I can maintain a lot of CRAFT archive. 


Yes, a lot more than if all images were uncompresed.
In fact, I was thinking of adding a script that'd keep only the last 24 hrs of Level II uncompressed, and then keep perhaps 10 days or more of compressed data as an archive. 


The current GEMPAK pqact.gempak_craft pattern action file has
the commented-out entry for decoding the CRAFT images.  This could
be turned back on and scouring could be setup to keep about a day
of uncompressed images.
Duh! <Slaps head> I should have noted that, but missed. it. I'll play there now. 


I'm also not seeing DDPLUS anywhere I can identify.



IDS|DDPLUS data is being received and apparently decoded.  There is
current data in:

/data/ldm/gempak/surface
/data/ldm/gempak/syn
/data/ldm/gempak/upperair
etc.

Where are you looking, or, what am I missing?
One of the projects I'm working on is using the text watches and warningsparsing same and placing the info on the described polygons in a postgis database. The .gem's don't get me there. I also was looking for the METARs...
I was going to start looking at making ShapeFiles from watch/warn data but I can't use the info here 'til I can find the DDPLUS stuff. 


Are you looking for the raw IDS|DDPLUS data to be written to disk, or
are you looking for the data to be decoded into GEMPAK format?  The
data is being decoded into GEMPAK format, so if something else is
missing, it is because there is no appropriate action(s) in
~ldm/etc/pqact.conf.  If this is the case, you could add the action
running on mesodata and send a HUP signal to the pqact that is
running to process actions in pqact.conf.


I'll take care of it.  THanks.
That's not a show-stopper. I'll nfs-mount from mesodata, but that box is creaking and may need to be forklifted soon. 


If you want to have a day's worth of uncompressed CRAFT data online,
let me know and I will turn the decoding back on.
If I don't need it for GPNEXR2 decoding, no. The compressed form is more efficient! 

Thanks, Gerry
--
Gerry Creager -- address@hidden
Texas Mesonet -- AATLT, Texas A&M University        
Cell: 979.229.5301 Office: 979.458.4020
FAX:  979.847.8578 Pager:  979.228.0173
Office: 903A Eller Bldg, TAMU, College Station, TX 77843