[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
20020110: 20010821: Gempak, Perl, and Gemplt errors
- Subject: 20020110: 20010821: Gempak, Perl, and Gemplt errors
- Date: Thu, 10 Jan 2002 16:12:49 -0700
Sean,
I'm off tp the AMS in Orlando, so can't look at your question
too closely.
As for the locking scripts I have, I touch a unique file with the process ID,
like .inuse.$$ ($$ is the csh process ID) but use the output from
"ls -rt | head -1" to get the oldest lock file. I don't rely on process
numbers increasing (since they will wrap around every so often).
eg:
set LOCK=.inuse.$$
touch $LOCK
@ COUNT = 0
set TEST=`ls -rt .inuse.* | head -1`
while(($TEST != $LOCK)&&($COUNT < 61))
sleep 10
set TEST=`ls -rt .inuse.* | head -1`
if($COUNT == 60) then
echo "Please check `hostname` on $TEST for nids generation $SITE $TYPE
$DATTIM" | \
/usr/bin/mailx -s NIDS chiz
rm $LOCK
exit 0
endif
@ COUNT = $COUNT + 1
end
In the case above, I limit the total number of times I will wait, and exit
if there appears to be a problem otherwise processes could build up.
Optionally, I can remove the old .lock file in the script and kill the process
in the script.
Steve Chiswell
Unidata User SUpport
>From: Sean Daida <address@hidden>
>Organization: UCAR/Unidata
>Keywords: 200201102255.g0AMtwN20059
>
>--------------36C73C074C006390299AFFF3
>Content-Type: text/plain; charset=us-ascii
>Content-Transfer-Encoding: 7bit
>
>Hi there Steve,
>
>Ok last time I hope.
>
>I was thinking about how the PID's were handed out and watched the processes
>some. I realized that most of the time the PID's were generally higher for th
> ose
>created later and lower for those created earlier. But this was not always th
> e
>case. I saw instances where a new process had a lower PID than an older one.
> If
>I understand the logic of my initial scripts sent to you, this would fool the
>script into allowing more than one gempak instance to run. For this to happen
> ,
>the older PID would have to pass the "while" statement and let the gempak prog
> ram
>run. While that gempak system call is processing, the newer request is made w
> ith
>a lower number PID. It, seeing itself at the top of the list, happily starts
>running a second instance of a gempak program. Even with this hole, I have fo
> und
>the scripts run from the cron are much more stable. Still, I felt that I shou
> ld
>make it stronger. The modification I added to the process lock script checks
> for
>the lock file modification time and sorts on that so that the processes are on
> ly
>allowed to run in the order of lock file creation, not by the order that the P
> ID's
>happen to be handed out in. Using the PID is still useful in maintaining
>uniqueness of each process. Does that make any sense? =)
>
>Anyway here is hopefully the last version of my perl scriptlette:
>
>#[previous stuff]
>
>#create process lock
>$count = 0;
>$lockdir = "/home/hokukea/rknabb/gempak/gemlock/tutt19";
>$pid = $$;
>$gemlock=".inuse.$pid";
>$PROCLOCK=">$lockdir/$gemlock";
>open(PROCLOCK);
>close(PROCLOCK);
>opendir DIRECTORY, $lockdir;
>@filelist = grep { /^\.inuse/ && -f "$lockdir/$_" } readdir DIRECTORY;
>closedir DIRECTORY;
>$count2=0;
>foreach $file(@filelist) {
> $filedate[$count2] = (stat($file))[9].$file;
> $count2++;
>}
>@filedate = sort(@filedate);
>while ( substr($filedate[0],10) ne $gemlock && $count < 61 ) {
> sleep 1;
> if ( $count > 60 ) { unlink("$lockdir/$gemlock") ; die; }
> @filelist = 0 ; @filedate = 0;
> opendir DIRECTORY, $lockdir;
> @filelist = grep { /^\.inuse/ && -f"$lockdir/$_"} readdir DIRECTORY;
> closedir DIRECTORY;
> $count2 = 0;
> foreach $file(@filelist) {
> $filedate[$count2] = (stat($file))[9].$file;
> $count2++;
> }
> @filedate = sort(@filedate);
> $count++;
>}
>
>#gempak program goes here!
>
>#clear process lock
>unlink("$lockdir/$gemlock");
>
>#[later stuff]
>
>Thanks again Seve and happy new year!
>Sean
>
>--------------36C73C074C006390299AFFF3
>Content-Type: text/html; charset=us-ascii
>Content-Transfer-Encoding: 7bit
>
><!doctype html public "-//w3c//dtd html 4.0 transitional//en">
><html>
>Hi there Steve,
><p>Ok last time I hope.
><p>I was thinking about how the PID's were handed out and watched the processe
> s
>some. I realized that most of the time the PID's were generally higher
>for those created later and lower for those created earlier. But
>this was not always the case. I saw instances where a new process
>had a lower PID than an older one. If I understand the logic of my
>initial scripts sent to you, this would fool the script into allowing more
>than one gempak instance to run. For this to happen, the older PID
>would have to pass the "while" statement and let the gempak program run.
>While that gempak system call is processing, the newer request is made
>with a lower number PID. It, seeing itself at the top of the list,
>happily starts running a second instance of a gempak program. Even
>with this hole, I have found the scripts run from the cron are much more
>stable. Still, I felt that I should make it stronger. The modifica
> tion
>I added to the process lock script checks for the lock file modification
>time and sorts on that so that the processes are only allowed to run in
>the order of lock file creation, not by the order that the PID's happen
>to be handed out in. Using the PID is still useful in maintaining
>uniqueness of each process. Does that make any sense? =)
><p>Anyway here is hopefully the last version of my perl scriptlette:
><p><tt>#[previous stuff]</tt>
><p><tt>#create process lock</tt>
><br><tt>$count = 0;</tt>
><br><tt>$lockdir = "/home/hokukea/rknabb/gempak/gemlock/tutt19";</tt>
><br><tt>$pid = $$;</tt>
><br><tt>$gemlock=".inuse.$pid";</tt>
><br><tt>$PROCLOCK=">$lockdir/$gemlock";</tt>
><br><tt>open(PROCLOCK);</tt>
><br><tt>close(PROCLOCK);</tt>
><br><tt>opendir DIRECTORY, $lockdir;</tt>
><br><tt>@filelist = grep { /^\.inuse/ && -f "$lockdir/$_" } readdir
>DIRECTORY;</tt>
><br><tt>closedir DIRECTORY;</tt>
><br><tt>$count2=0;</tt>
><br><tt>foreach $file(@filelist) {</tt>
><br><tt> $filedate[$count2] = (stat($file))[9].$file;</tt>
><br><tt> $count2++;</tt>
><br><tt>}</tt>
><br><tt>@filedate = sort(@filedate);</tt>
><br><tt>while ( substr($filedate[0],10) ne $gemlock && $count <
>61 ) {</tt>
><br><tt> sleep 1;</tt>
><br><tt> if ( $count > 60 ) { unlink("$lockdir/$gemlock") ; die;
>}</tt>
><br><tt> @filelist = 0 ; @filedate = 0;</tt>
><br><tt> opendir DIRECTORY, $lockdir;</tt>
><br><tt> @filelist = grep { /^\.inuse/ && -f"$lockdir/$_"}
>readdir DIRECTORY;</tt>
><br><tt> closedir DIRECTORY;</tt>
><br><tt> $count2 = 0;</tt>
><br><tt> foreach $file(@filelist) {</tt>
><br><tt> $filedate[$count2] = (stat($file))[9].$file;</tt>
><br><tt> $count2++;</tt>
><br><tt> }</tt>
><br><tt> @filedate = sort(@filedate);</tt>
><br><tt> $count++;</tt>
><br><tt>}</tt>
><p><tt>#gempak program goes here!</tt>
><p><tt>#clear process lock</tt>
><br><tt>unlink("$lockdir/$gemlock");</tt>
><p><tt>#[later stuff]</tt><tt></tt>
><p><tt>Thanks again Seve and happy new year!</tt>
><br><tt>Sean</tt></html>
>
>--------------36C73C074C006390299AFFF3--
>
