Jeffrey Lake - Admin wrote:
Sorry on the first reply Gilbert going directly to you..
Anybody got idea's how to implement this on CentOS ??
I have the available memory (4GB) but it hardly seems to be used
-Jeff
Jeff,
I just checked on several of my Centos/RHEL machines, and the tmpfs
appears to be enabled by default on RHEL 3/4 and Centos 4/5 (at least
those are the ones I checked)
For example, on my primary ingest/relay machine idd.aos.wisc.edu, it's a
dual opteron w/8Gb of RAM, and I didn't do anything special, but doing a
df shows
[root@idd ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
72G 17G 52G 24% /
/dev/sda1 99M 74M 21M 79% /boot
none 3.9G 0 3.9G 0% /dev/shm
I may play around with this too. Probably on local machines first so I
don't impact people downstream if I mess up!
Pete
Hello all,
I'm just curious as to how many people run their LDM queues from memory?
Using Fedora Core 7 or 8, the default kernel sets aside memory in a device
known as "/dev/shm". Do a "df -h" and you'll see it. Depending on the size
of your physical memory, the default kernel setting gives you about 25% of
the available memory to put files in as an extra "hard drive", if you
will. Compiling your own kernel, you can put in as little or as much as
you want. On my system, I have the following:
% df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
707523888 25768472 645235480 4% /
/dev/sda1 101086 18576 77291 20% /boot
tmpfs 1815312 1197764 617548 66% /dev/shm
To run the LDM queues out of memory, you need to put your ldm.pq and
pqsurf.pq files into /dev/shm, and remake them every time you reboot,
since /dev/shm is really just memory. The easiest way to
do this with LDM 6.6.X or 6.7.X.X is to go into your ldmadmin-pl.conf
file in your ~/etc directory with the changes I have in mine:
$bin_path = "$ldmhome/bin";
$etc_path = "$ldmhome/etc";
$log_path = "$ldmhome/logs";
$data_path = "$ldmhome/data";
#$pq_path = "$data_path/ldm.pq";
#$surf_path = "$data_path/pqsurf.pq";
$pq_path = "/dev/shm/ldm.pq";
$surf_path = "/dev/shm/pqsurf.pq";
Note the changes to $pq_path and $surf_path.
Now, when I go to /dev/shm, this is what I see:
% cd /dev/shm
% ls -al
total 1197764
drwxrwxrwt 2 root root 80 2008-02-14 10:22 .
drwxr-xr-x 12 root root 4060 2008-02-14 11:31 ..
-rw-rw-r-- 1 ldm users 1222955008 2008-02-14 10:22 ldm.pq
-rw-rw-r-- 1 ldm users 2347008 2008-02-14 10:22 pqsurf.pq
Since the LDM queues run in memory, you can make a 1.2 GB ldm.pq in just a
few seconds on a Core 2 processor. Much more importantly, if you have a
lot of feeds and/or are a relay site, this cuts down on your disk
read/writes by a TON. For instance, I am getting every Level 2 data site
minus Alaska and NOP (still trying to figure out what ldmd.conf request
line I need to get the latter test site), and my load average is around
.5. before this, it was around 3 on "light" days, but when there's a lot
of stuff happening around the country, it used to go up to 12! Now, it's
been cut by a factor of 6. That's HUGE, and it saves my hard drive, too.
My overall load average has dropped by 75% or so since I have done this.
And thanks to Dave Bukowski at the College of DuPage for suggesting this,
as well as Tom Yoksas from UNIDATA and Mike Dross from Wright-Weather.
It really works!
Thoughts?
*******************************************************************************
Gilbert Sebenste
********
(My opinions only!) ******
Staff Meteorologist, Northern Illinois University ****
E-mail: sebenste@xxxxxxxxxxxxxxxxxxxxx ***
web: http://weather.admin.niu.edu **
*******************************************************************************
_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
--
+>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>+<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<+
^ Pete Pokrandt V 1447 AOSS Bldg 1225 W Dayton St^
^ Systems Programmer V Madison, WI 53706 ^
^ V poker@xxxxxxxxxxxx ^
^ Dept of Atmos & Oceanic Sciences V (608) 262-3086 (Phone/voicemail) ^
^ University of Wisconsin-Madison V (608) 262-0166 (Fax) ^
+<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<+>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>+
