[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20000502: Using pq_insert when LDM isn't running (fwd)

Gregory Grosshans wrote:

> Anne Wilson wrote:
>
> > ...text deleted...
> >
> > There is a related issue, however, that may or may not be of interest to 
> > you.
> > When you start the acqserver then start the LDM, the resulting LDM processes
> > belong to one process group and the acqserver belongs to another.  The LDM
> > processes communicate by sending signals amongst themselves; this depends 
> > on them
> > belonging to the same process group.  Of particular interest here, when a 
> > new
> > product is inserted into the queue a signal is sent to inform other 
> > processes.
> > Your 'configuration' still works because processes will also poll to see if 
> > the
> > queue has changed.  However, this is inefficient ( the technical term 
> > 'clunky' has
> > been used here).  If efficiency is not a problem for you, great!  If it is, 
> > then
> > we'll need to talk further.   FYI.
>
> Normally I start LDM as user ldm and then start the acqserver as user ldm.  
> Is there
> significant performance impacts when the queue is .75 GB?  Also, what would be
> involved in making the acqserver be part of the process group under LDM?
>
> Thanks for the good news, I'm just interested in these additional details.
>
> Gregg

The performance issues I brought up here are not affected by the queue size.

Who you are when you start the acqserver and the LDM affects who 'owns' the 
resulting
processes, but does not affect what process group those processes belong to.
Typically processes are grouped into the same process group by virtue of being 
spawned
by a parent process or a descendent of the parent process.  Since the LDM 
spawns all
those processes, they all belong to the same group.

You could have the LDM start your acqserver by putting an additional exec 
command in
your ldmd.conf file that would spawn the acqserver.    Then  the pqinsert call 
in the
acqserver would notify the other LDM processes of queue changes by means of a 
signal.
This way other LDM processes would be informed immediately of the change in the 
queue,
rather than waiting until polling occurs to find out.  (The polling may or may 
not
continue to be enabled in this case - I'd have to do some research to find 
out.)  The
problem with this is that when you shut down the LDM, the acqserver might be 
shut down
as well since child processes are often killed when their parent dies.  So you 
might
still miss some data.

Another possibility would be to run an LDM upstream on your NOAAPort receiver 
system,
which would give you an hour's worth of buffer, which may or may not be 
feasible for
you.

But, I guess my real question is (which I should have asked yesterday), why are 
you
stopping the LDM to copy your metar file?   You can get it out of your own 
queue by
using 'pqcat' or you can have your LDM dump it to stdout when it arrives using
'feedme'.  Would either of these solve your problems?  If so, then you could 
have the
LDM start your acqserver thereby reaping the improved efficiency and you would 
not need
to shut anything down.

Anne

--
***************************************************
Anne Wilson                     UCAR Unidata Program
address@hidden                  P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************