[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[TIGGE #EEC-237791]: Memory problem with LDM Server at CMA

MA Qiang,

> I will apply for more memory installed on LDM server .
> Could you please give me some advice on how many memory I should add?

You should have enough memory so that the entire LDM product-queue
and all the LDM processes can be held in memory without swapping.
Because the product-queue is memory-mapped and shared, it will occupy
only one area in memory.  The code portions of the "rpc.ldmd" processes
are also shared.  As an initial guess, I suggest that you have twice
as much memory as the size of the product-queue (this is probably an
overestimate).

You might ask others in the TIGGE project what they have.

> Can I calculate the memory needed by LDM from the size of the queue?

Only in very general terms, because it also depends on the number of
LDM processes and other activities on the computer.  The primary 
criteria is to prevent swapping.

> Should I increase the size of LDM product-queue to 14378MB(=2560M*3600/641),

Yes.

> and increase physical memory over 45GB(=8GB*3600/641)?

No, because the product-queue is shared.  Twice as much memory as the
size of the product queue would come to about 28 GB.

> 2560M is the size of LDM product-queue and 8GB is the amount of memory now.

Yes, the product-queue is currently too small.  It also is limited by the
size of the data portion of the queue: it has many more slots for products
than can be used -- given the average size of the TIGGE data-products.  I
suggest that the number of slots be twice the maximum number of products
that could arrive in one hour.

Look at the file ~/logs/tigge-ldm.uptime.  It can help you monitor your
LDM system.  The fields are:

YYYYMMDD.hhmm time
load_1 1-minute load average
load_5 5-minute load average
load_15 15-minute load average
outbound# Number of outgoing connections
inbound# Number of incoming connections
total# Total number of connections
age Age in seconds of the oldest product in the product-queue
memFree Amount in bytes of free memory
swapUsed Amount in bytes of swap space used
user% Proportion of CPU time in percent spent in user-mode
sys% Proportion of CPU time in percent spent in system-mode
iowait% Proportion of CPU time in percent spent in I/O-wait
idle% Proportion of CPU time in percent spent idling

The "age" parameter is the important one for determining the
size of the product-queue.  Ideally, it should always be
greater than 3600.

I tried to plot the above information for you, but I couldn't
configure ssh to automatically allow me to copy the "uptime"
file.

For additional information, see
<http://www.unidata.ucar.edu/software/ldm/ldm-current/basics/vet-ldmadmin.html>.

> Thank you!
> 
> Best regards,
> MA Qiang
> 
> National Meteorological Information Center
> 2008-01-17

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: EEC-237791
Department: Support IDD TIGGE
Priority: Normal
Status: On Hold