[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20000216: [Fwd: LDM version 5.0.9] (fwd)

>
> Date: Wed, 16 Feb 2000 08:37:36 -0700
> From: Unidata Support <address@hidden>
> Reply-To: Jeff Ator <address@hidden>
> To: address@hidden
> Subject: 20000216: [Fwd: LDM version 5.0.9]
>
> >To: address@hidden
> >From: Jeff Ator <address@hidden>
> >Subject: [Fwd: LDM version 5.0.9]
> >Organization: .
> >Keywords: 200002161347.GAA24766
>
> This is a multi-part message in MIME format.
> --------------974A8FFEAD727DD0DC8AA4DA
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
>
> Here is some further information that might help you diagnose the
> problem in #1 below.  Every time pqexpire runs (at 51 minutes after each
> hour, via cron) it produces the following output:
>
> Feb 14 19:51:00 nco3n03 pqexpire[21594]: Starting Up
> Feb 14 19:52:08 nco3n03 pqexpire[21594]: Exiting
> Feb 14 19:52:08 nco3n03 pqexpire[21594]: > Up since:
> 20000214195100.898
> Feb 14 19:52:08 nco3n03 pqexpire[21594]: > Queue usage (bytes):28222800
> Feb 14 19:52:08 nco3n03 pqexpire[21594]: >          (nregions):   48895
> Feb 14 19:52:08 nco3n03 pqexpire[21594]: > nprods deleted 0
>
> The thing to note is that the "Queue usage" values that are printed, in
> terms of both bytes and nregions, stay unchanged with each hourly run of
> pqexpire!?  This seems to indicate that only around 28.2Mb of the
> available queue size are ever being used, even though (as explained in
> the original message), the queue was originally allocated with 100Mb of
> space.  This would explain why we are seeing the "Deleting oldest to get
> a queue slot" messages after only 6 hours (and why pqexpire thus has
> nothing to delete if it is only looking to purge products older than 12
> hours!), but the question then becomes "Why is only such a small
> percentage of the available queue space being utilized?".
>
> I would appreciate any insight you can provide on this perplexing
> issue.  Thanks very much!
>
> Sincerely,
> Jeff
>
> --
> __________________________________
>
> Jeffrey B. Ator
>
> National Centers for
>    Environmental Prediction
> Central Operations
> 5200 Auth Road, Rm. 307
> Camp Springs, MD  20746
>
> Phone:  (301) 763-8000, Ext. 7104
> Fax:    (301) 763-8381
>
> E-Mail:  address@hidden
>
> __________________________________
>
> --------------974A8FFEAD727DD0DC8AA4DA
> Content-Type: message/rfc822
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline
>
> X-Mozilla-Status2: 00000000
> Message-ID: <address@hidden>
> Date: Tue, 15 Feb 2000 17:54:56 -0500
> From: Jeff Ator <address@hidden>
> X-Mailer: Mozilla 4.5 [en] (X11; I; Linux 2.0.36 i686)
> X-Accept-Language: en
> MIME-Version: 1.0
> To: address@hidden
> Subject: LDM version 5.0.9
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
>
> Hello,
>
> We have been running LDM version 5.0.5 on an ORIGIN 200 (IRIX64)
> platform and are now moving our data ingest processing to an AIX 4.3
> platform.  We have downloaded and installed LDM version 5.0.9 on this
> new machine and have noticed a few differences that we could not find
> explanations for in the RELEASE NOTES nor in the online support archive:
>
> 1.  The queue size needed to store the same amount of incoming data
> seems to have dramatically increased between the two versions.  Under
> 5.0.5 on the IRIX64, we used to be able to store more than 12 hours of
> incoming data in an ldm.pq file that was around 75Mb in size.   Now,
> under 5.0.9 on AIX4.3, we can't even store the same amount of data
> within a 100Mb queue file.  In fact, with this larger queue size we can
> only run pqing for about 6 hours before we start seeing the pqing
> diagnostic "Deleting oldest to get a queue slot" for each new incoming
> product.  Is there some explanation for this?  On the old system, we ran
> pqexpire from cron with the "-a 12" option to always keep the most
> recent 12 hours worth of data in the queue.
>
> 2.  I've found that when running "ldmadmin mkqueue" on AIX4.3, the -s
> option to specify a queue size is not honored.  Rather, the default size
> specified within the "ldmadmin" script itself seems to take precedence.
> I always thought that the -s option for mkqueue took precedence.
>
> Thanks for any assistance you can provide!
>
> Sincerely,
>
> --
> __________________________________
>
> Jeffrey B. Ator
>
> National Centers for
>    Environmental Prediction
> Central Operations
> 5200 Auth Road, Rm. 307
> Camp Springs, MD  20746
>
> Phone:  (301) 763-8000, Ext. 7104
> Fax:    (301) 763-8381
>
> E-Mail:  address@hidden
>
> __________________________________

Hi Jeff,

A brief background: the LDM queue is a memory mapped file, using the mmap
function.  mmap is pretty low level and thus system dependent.  Also, AIX (and
Linux) allocate memory differently than the other OSs we support - they allocate
in 'regions'.

Not many of our users use AIX.  It's entirely possible that there's a problem
related to
mmap under AIX.  One test you could run for us would be to load 5.0.9 on your 
IRIX
machine.   Presumably it would run fine, and we could rule out your 
configuration
as a problem.  Would you be willing to do that?

Anne

--
***************************************************
Anne Wilson                     UCAR Unidata Program
address@hidden                  P.O. Box 3000
(303) 497-8677                    Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************