[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 19990615: ldm pq

> >
> >This has come up before and I wonder if there is a solution.
> >There are times when I want to make an existing LDM product queue a
> >little bit larger. The only way I know to deal with this is to
> >re-create the queue at the larger size (ldmadmin mkqueue -c) but
> >this destroys the existing queue and all data in it. So upon restarting
> >the ldm, you get all the last hours data _again_ from the upstream
> >source
> >which I don't want. An alternative is to edit the ldmadmin and set
> >the rpc.ldmd -o option to 0 to prevent it from getting those old
> >products. But that's cumbersome, since normally I want to leave
> >that alone inside the ldmadmin script. Is there any way of resizing
> >the existing queue (without destroying data) or copying the old queue
> >contents to the new queue?
> >
> >Thanks in advance,
> >
> >-jim

> 
> Jim,
> 
> The LDM product queue can grow as needed, however, the configure script
> defines NO_GROW for several platforms including solaris. On Irix, which
> I run the Conduit data feed on, the default is to allow the queue to grow.
> The downside of having a growable queue is that there is no high water mark-
> and its possible that when pqexpire is running and the queue is growing,
> the lock on the queue can really get slow. In general, I create the queue as 
> large
> as I expect to need- so the queue won't have to grow often- but if a need
> should arise, then it is possible. I wouldn't suggest creating a default 35mb
> queue and have the data push it up to its normal running level though
> sinceyou will almost definitely run into conflict with pqexpire if
> you are growing that much. Anyhow, you can grow the queue but its a compile
> time option. If you mainly need to resize the queue on a quiescent system
> it should be OK.

Thanks Chiz, we're running the LDM on both IRIX and Solaris. I noticed a 
couple of weeks ago that our IRIX ldm's product queue did grow in
response
to some new data, but it really brought that system to its knees.
Combined
with the fact that the default on Solaris is to NOT grow the queue
dynamically,
I thought I'd just try to aviod that feature altogether. 

The thing that always seems to happen is I pick a size I think I won't
exceed, and then along comes a new data feed that causes me to grow it
again. Then when I create the new queue I get all that old data from
upstream. I was just trying to find an easy way to avoid duplicating
that
data. What would be nice would be to have a 'pqresize' switch in
ldmadmin
that basically made a new queue (at the larger size), copied the
contents
of the old queue into the new one, and then moved the new queue into
place.
Of course it would have to stop the ldm, keep a temp file for the new
queue,
do the file moves and then restart the ldm again to do this right. I
don't
know how easy it would be to move the products between the queues but I
know if there was a feature like this I would use it.

-jim