[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Change program name to kudzu?

"James R. Frysinger" wrote:
> 
> Good morning!
> 
> When it rains it pours. I seem to have solved some of the problems that
> were keeping data from being decoded and stored on our local computer,
> weather.cofc.edu. Now, the user hard drive partition keeps filling up --
> even after an "ldmadmin stop". It seems that stopping ldm doesn't stop
> data ingestion!
> 
> The pqact.conf file has the standard WMO data line that was included in
> the distribution. I'm planning on modifying it to limit the WMO data
> input, probably by specifying only "US" for the 3rd and 4th characters
> (and as appropriate for Canada, the Gulf, etc.). I expect that at first,
> at least, North American area data will be of the greatest interest to
> our users. More could be added later in additional lines, I suppose.
> 
> The pqact file also has the standard MCIDAS request lines, cut and
> pasted from the sample web page specified in the ldm manual. I
> installed the ldm-mcidas decoder and provided a symbolic link to its bin
> address in the decoder section of ldm. That stopped the problem of
> getting frequent error messages about pnga2area not existing. I'll do
> the same with SATANNOT and SATBAND in the etc directory of ldm-mcidas
> to clear the errors I'm getting on those. (Or just copy them into the
> decoder section.) Fortunately, your archived emails have been very
> educational --- to the point where I'm probably dangerous, as indicated
> by my shrinking free disk space. ;-)
> 
> On Saturday, after spending some time doing the tutorial in the MCIDAS
> Learning Guide on using real time data (during which I downloaded,
> saved, and executed the RTSERVER.BAT file) I discovered this problem
> with the user partition on the hard disk of weather.cofc.edu filling up
> completely! I stopped ldm with "ldmadmin stop", revised my scour.conf
> file to add a line
>    ~ldm/data    1               *.wmo
> to keep my .wmo files to a minimum and I managed to recover about a
> gigabyte of space by manually initating scour. It's filling back up,
> though, even though ldm is supposedly not running! Also, this makes the
> scour routine hard to complete in a 20 minute window!
> 

On Mon, 15 Jan 2001, James R. Frysinger wrote:
> Good morning!
....
> though, even though ldm is supposedly not running! Also, this makes the
> scour routine hard to complete in a 20 minute window!
/***brain spasm--->> scour is on a 3 h cycle, not 20 min.***/



> Can you give me some ideas on what else to look for to control this
> "in-leak" of data? Is MCIDAS  bringing in data via ADDE on its own?
> (That's not what I glean from the manual, though.) How can I control my
> disk usage better? What is a reasonable size for the data storage needs
> of WMO and MCIDAS data? Any comments on my actions and intentions above?
> 
> Another thought... Right now, my WMO data is going into
>    /export/home/mcdata/ldm/
> and I also have the following directories in there:
>   /export/home/mcdata/ldm/campus/
>    /export/home/mcdata/ldm/decoded/
>    /export/home/mcdata/ldm/forecasts/
>    /export/home/mcdata/ldm/gempak/
>    /export/home/mcdata/ldm/ldm/
>    /export/home/mcdata/ldm/logs/
>    /export/home/mcdata/ldm/mcidas/
>    /export/home/mcdata/ldm/severe/
>    /export/home/mcdata/ldm/surface/
>    /export/home/mcdata/ldm/upperair/
>    /export/home/mcdata/ldm/warnings/
> as well as my ldm.pq (500 MB). Most of those directories are empty
> except for the appropriate .scour file. Any suggestions on a better way
> to organize my data directories would be appreciated.
> 
> I've added my sysadmin-ldm address above, since that's the easiest
> account to read when I'm on the console locally. Please include that as
> well as my regular (frysingerj) address.
> 
> Don't you hate Mondays? Sorry....
> 
> Jim
> 
>  --
> James R. Frysinger                  University/College of Charleston
> 10 Captiva Row                      Dept. of Physics and Astronomy
> Charleston, SC 29407                66 George Street
> 843.225.0805                        Charleston, SC 29424
> http://www.cofc.edu/~frysingj       address@hidden
> Cert. Adv. Metrication Specialist   843.953.7644
"James R. Frysinger" wrote:
> 
> Partial answer?... When I execute "ldmadmin stop" it appears that the
> crontabbed ldmfail kicks in to restart rpc.ldmd, which of course sires
> pqact. This sounds like a "shoot yourself in the foot" configuration.
> Could it be that I have things set up wrong? I thought ldmadmin worked
> off of ldmping and not on whether data was coming in or not.
> 
> Jim
> 

Morning, Jim!

So now you have TOO MUCH data!!! :-)  

First, you should always confirm that the ldm has actually stopped when
you do 'ldmadmin stop'.  Do 'ldmadmin ps' and also do 'ps -ef | grep
ldm' to verify that all the ldm processes have died.  

But, keep in mind that after stopping the ldm it is very common for ldm
processes to hang around for a while.  If an rpc.ldmd process is writing
to the queue or transmitting a product, it will finish its task then die
gracefully.  Similar for pqact.  If a process is working on a large
product or if there's a slow connection this could take a while.  For
this reason we tell people to always do 'ldmadmin ps' and 'ps -ef | grep
ldm' before restarting the ldm.  If you restart the ldm but still have
some old processes hanging around from a previous invocation it could
get into a confused state.  

This makes me wonder if you have some rogue processes left over from
previous invocations of the ldm. 'ps -ef | grep ldm' will give you the
parent PID of each process.  If you see more than one parent for your
processes then you have rogue processes that probably should be
assasinated.  

pqact is the program that files products to the disk and is thus part of
this problem you're having.  If somehow pqact is still running it will
continue processing everything in the queue until all products have been
processed, then it will idle until more products become available. 
Could you have a rogue pqact process running?  Still, there would need
to be a rpc.ldmd process running to be putting more products into the
queue.

(FYI, if you must annihilate an rpc.ldmd process, thus not allowing it
to die gracefully, you may corrupt your product queue.  In this case I
would recommend deleting the queue and remaking it.  The cost is having
to retrieve the past hour's worth of data, which is usually bearable.)

Anyway, in summary, the ldm should stop when you tell it to.  You can
and should confirm that all processes are stopped after you stop the
ldm.  This should stop all requests for data, and also stop pqact so
that no products are being written to the disk.

I'm sorry but I can't answer McIDAS questions for you.  Our McIDAS
person is at AMS this week, but if you send your McIDAS questions to
support@unidata, someone will try to help you out.   Having said this, I
don't think that the ADDE server will be retrieving data unless you
iniate that operation.

Also, it is the case that if ldmfail is invoked while the ldm is down it
will attempt to restart it. But, its invocation is not linked to
'ldmadmin stop'.  If you've scheduled ldmfail to run every 20 minutes
and your ldm is down longer than that, then ldmfail will be invoked and
try to restart it.  Perhaps you should comment out that line in your
crontab until you get this other stuff straightened out. 

Hope this helps!

Anne
-- 
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                 P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************