If no one else mentioned this, you're a few updates (and useful ones)
behind. Install ldm-6.10.1 at your next opportunity.
This also sounds like you're trying to process a LOAD of stuff via
pqact. I suspect you're getting bogged down with io requests.
The link you've found is a good one, and the -o 1 trick is one I've used
in the past when I was trying to restart after problems.
gerry
On 03/20/2012 03:36 PM, Adam DeConinck wrote:
Thanks all for the quick replies!
We seem to be recovered for the moment, and will be revisiting our
queue configuration. We may also be imposing limits using ulimit after
a quick review, but I was trying to determine if LDM had any built-in
throttling mechanism available. I always prefer to let the
application limit itself if possible before letting the OS beat on it.
We also found the following thread that has potentially useful things
to say about this type of issue:
(http://www.unidata.ucar.edu/support/help/MailArchives/idd/msg05853.html)
I will have to determine from the users of the data whether limiting
the amount of the backlog to download, with -o, is acceptable from
their perspective.
Thanks again,
Adam
On Tue, Mar 20, 2012 at 3:24 PM, Patrick L. Francis
<wxprofessor@xxxxxxxxx> wrote:
reduce the size of your product queue, delete and
rebuild your queue, then adjust ldmd.conf to only
one feed such as IDS-DDSPLUS to diagnose issues...
setting limits should not be needed, but if you feel
strongly about it you might want to google 'ulimit'
cheers,
--patrick
Hi all,
We're currently attempting to diagnose a problem on an LDM node which
we believe has a major backlog of data to download. When we attempt
to start LDM, it quickly takes over all available memory and the load
skyrockets, until the server chokes and dies. Server has 8 cores and
32 GB memory, and 8 GB swap. As of right now, the load is over 600
and nearly all memory is in use.
We are not sure exactly what the cause of this issue is in LDM... only
that as soon as we start LDM, the load goes through the roof.
This issue occurred once before, which we solved with a memory upgrade
8GB->32 GB. Now it's happening again with more memory. :P
LDM version is 6.8.1.
Can anyone suggest a way to instruct LDM to rate-limit itself, or
otherwise set a resource limit so it doesn't murder the server it's
on?
Thanks,
Adam
