[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
Re: 19991103: LDM 5.0.8 and 'expanding input buffer size' problemwithAFOS feed (fwd)
- Subject: Re: 19991103: LDM 5.0.8 and 'expanding input buffer size' problemwithAFOS feed (fwd)
- Date: Fri, 5 Nov 1999 10:34:45 -0700 (MST)
pqing.diff for afos
Index: pqing.c
===================================================================
RCS file: /upc/share/CVS/ldm5/pqing/pqing.c,v
retrieving revision 1.65
retrieving revision 1.66
diff -c -r1.65 -r1.66
*** pqing.c 1998/10/16 19:28:49 1.65
--- pqing.c 1999/02/09 03:17:09 1.66
***************
*** 2,10 ****
* Copyright 1993, University Corporation for Atmospheric Research
* See ../COPYRIGHT file for copying and redistribution conditions.
*/
! /* $Id: pqing.c,v 1.65 1998/10/16 19:28:49 steve Exp $ */
static char version[] =
! "$Revision: 1.65 $ built "__DATE__" "__TIME__;
#include <ldmconfig.h>
#include <stdio.h>
--- 2,10 ----
* Copyright 1993, University Corporation for Atmospheric Research
* See ../COPYRIGHT file for copying and redistribution conditions.
*/
! /* $Id: pqing.c,v 1.66 1999/02/09 03:17:09 davis Exp $ */
static char version[] =
! "$Revision: 1.66 $ built "__DATE__" "__TIME__;
#include <ldmconfig.h>
#include <stdio.h>
***************
*** 512,525 ****
else
setTheScanner(scan_wmo_binary);
}
- else if (feedtype & NMC)
- {
- setTheScanner(scan_wmo_binary);
- }
else if (feedtype == AFOS)
{
prod_stats = afos_stats;
setTheScanner(scan_afos);
}
else if (feedtype == FAA604)
{
--- 512,525 ----
else
setTheScanner(scan_wmo_binary);
}
else if (feedtype == AFOS)
{
prod_stats = afos_stats;
setTheScanner(scan_afos);
+ }
+ else if (feedtype == NMC2) /* CONDUIT */
+ {
+ setTheScanner(scan_wmo_binary);
}
else if (feedtype == FAA604)
{
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================
---------- Forwarded message ----------
Date: Fri, 05 Nov 1999 17:19:54 +0000
From: Gregory Grosshans <address@hidden>
To: Robb Kambic <address@hidden>
address@hidden, address@hidden
Subject: Re: 19991103: LDM 5.0.8 and 'expanding input buffer size'
problemwithAFOS feed
Thanks for the patch! LDM has been happily ingesting AFOS data since late
yesterday afternoon.
Gregg
Robb Kambic wrote:
> Gregory,
>
> I actually found the patch file, added it as a attachment. Here's the tag
> on how to apply it:
>
> http://www.unidata.ucar.edu/glimpse/ldm/2821
>
> According to the patch, pqing.c 1.66 should work, then the PIL code can be
> added. I'll attach 1.66
>
> I forgot about David Copley problem, here's logs for pqing.c. It looks
> like the PIL code was added after NMC2 stuff.
>
> ----------------------------
> revision 1.68
> date: 1999/06/17 16:11:07; author: steve; state: Exp; lines: +3 -2
> Replaced missing "#include <ldmconfig.h>" directive that was removed at
> the last checkin.
> ----------------------------
> revision 1.67
> date: 1999/06/03 21:12:38; author: rkambic; state: Exp; lines: +27 -8
> PIL header addition
> ----------------------------
> revision 1.66
> date: 1999/02/09 03:17:09; author: davis; state: Exp; lines: +6 -6
> Fix CONDUIT hack.
> ----------------------------
> revision 1.65
> date: 1998/10/16 19:28:49; author: steve; state: Exp; lines: +3 -2
> Ported to HP-UX B.11.00.
> Modified configuration mechanism: now uses configuration file
> "ldmconfig.h.in".
> ----------------------------
>
> You might have add the line to pqing.c:
>
> #include <ldmconfig.h>
>
> Robb...
>
> On Thu, 4 Nov 1999, Gregory Grosshans wrote:
>
> > Robb,
> >
> > I tried a feedtest and received the following:
> >
> > bin/feedtest -vxl - -b 9600 -p none -f AFOS /dev/ttyC13
> > Nov 04 21:09:31 feedtest[23658]: Starting Up
> > $Revision: 1.68 $ built Aug 13 1999 11:23:12
> > Nov 04 21:09:31 feedtest[23658]: TERMIOS "/dev/ttyC13": 9600 baud, none
> > parity
> > garbage encountered while searching for header
> > Nov 04 21:10:03 feedtest[23658]: Expanding input buffer size to 20480
> > Nov 04 21:10:13 feedtest[23658]: Expanding input buffer size to 24576
> > Nov 04 21:10:16 feedtest[23658]: Interrupt
> > Nov 04 21:10:16 feedtest[23658]: Exiting
> > leslie 47:
> >
> > So I looked through the Unidata searchable web pages using 'afos'. I found
> > some email from
> > this past February between Unidata and David Copley. He appeared to have
> > the same problem
> > as I am only he was using 5.0.6.
> >
> > I called David and he said that Unidata noticed a bug in pqing.c and that
> > Unidata provided a
> > patch and then LDM ingested AFOS
> > data just fine. Looking at some other email it appears pqing was updated
> > to support the
> > CONDUIT project. Can you provide me the patch necessary for pqing to
> > accept AFOS data
> > instead of the CONDUIT NMC2 data?
> >
> > Thanks,
> > Gregg
> >
> >
> > Robb Kambic wrote:
> >
> > > Gregory,
> > >
> > > I don't have any good solutions to your problem. I created an source code
> > > release for ldm-5.0.2 in the standard ldm source ftp directory. You could
> > > compile the source, test it and then modify the PIL fixes to the code. One
> > > word of caution, save all the other working releases so no overwrites
> > > occur. At this time, I can't really expend any more time because AFOS
> > > stream is not really used in the IDD. If I think of any new ideas about
> > > the problems, I'll let you know. I had numerous problems with HPUX, if
> > > fact I can't get it compiled on HPUX 11.0 because of a bug in the rpc
> > > library. For HPUX 11.0 users, they have to use the version compiled on
> > > HPUX 10.2 I sympathize with your situation, it's frustrating.
> > >
> > > Robb...
> > >
> > > On Thu, 4 Nov 1999, Gregory Grosshans wrote:
> > >
> > > > I'm running LDM 5.0.2, and trying to get 5.0.8 to work on an HP 712
> > > > HPUX 10.20
> > > > machine. Up until this past spring the LDM was running on an HP 715
> > > > with HPUX 9.x.
> > > > The feeds into this machine are AFOS, DDS and PPS via serial
> > > > connections.
> > > >
> > > > I just grabbed the HPUX 10.2 LDM 5.0.8 binaries off the web site and
> > > > installed them and
> > > > set the queue at 100 MB. Again I received
> > > > the "Expanding input buffer size" problem.
> > > >
> > > > I've attached ldmd.log.1 and ldmd.log.2. ldmd.log.1 was created when I
> > > > tried to run
> > > > the 5.0.8 binaries and you will notice the
> > > > expanding buffer size messages. The interesting item at the end of
> > > > this log is under
> > > > afos it lists 'nregions 35'. While the PPS and
> > > > DDS feeds don't list an nregions at all. Then if you look at the end
> > > > of ldmd.log.2
> > > > (created from 5.0.2) under AFOS it doesn't list
> > > > 'nregions...'. Do you know what causes LDM to generate this message?
> > > >
> > > > Outside of just a few HP's with Mux ports the other alternatives are
> > > > configuring a
> > > > linux box. Operations is just an HP shop.
> > > >
> > > > The router configuration would involve NCEP HQ so I'd like to wait
> > > > until absolutely
> > > > necessary to go that route.
> > > >
> > > > Gregg
> > > >
> > > >
> > > >
> > > > Robb Kambic wrote:
> > > >
> > > > > Gregory,
> > > > >
> > > > > At this point, I'm really starting to speculate. Is it possible to
> > > > > install
> > > > > the ldm on another box that's not IRIX? My thinking is some
> > > > > library/etc
> > > > > has changed to make the ldm behave abnormal. If we could duplicate
> > > > > this on
> > > > > another platform then it's definitely a bug in the ldm. Could be OS
> > > > > specific.
> > > > >
> > > > > Robb...
> > > > >
> > > > > On Thu, 4 Nov 1999, Gregory Grosshans wrote:
> > > > >
> > > > > > On the machine in question the only feeds are DDS, PPS and AFOS.
> > > > > > On this machine
> > > > > > I've had a LDM queue of 25 MB for three years and it worked fine
> > > > > > with all three
> > > > > > feed types.
> > > > > >
> > > > > > There is a seperate machine with LDM ingesting NOAAPORT and on this
> > > > > > machine the
> > > > > > queue is 700+ MB since I'm ingesting three
> > > > > > channels of NOAAPORT data.
> > > > > >
> > > > > > I've actually set the size of the ldm queue in the
> > > > > > $LDMHOME/bin/ldmadmin script to
> > > > > > 250000000. I've verified several times that the queue is of
> > > > > > sufficient length.
> > > > > > When I switch back to 5.0.2 I change the queue back to 25 MB and
> > > > > > everything works
> > > > > > fine.
> > > > > >
> > > > > > I tried commenting out the PPS and DDS feed types, leaving only
> > > > > > AFOS on and
> > > > > > starting 5.0.8 and still received the 'expanding input buffer
> > > > > > size'. While LDM
> > > > > > was running with just the AFOS feed I tried pqcat. PQCAT started
> > > > > > and exited right
> > > > > > away and indicated the number of products were 0.
> > > > > >
> > > > > > Gilbert, If I were able to have the routers and firewall changed to
> > > > > > allow your LDM
> > > > > > access to this machine I don't understand how the AFOS
> > > > > > data would be able to get to you. Right now there are no AFOS
> > > > > > products entering
> > > > > > the queue, the only thing occurring is the 'Expanding input buffer
> > > > > > size.' If
> > > > > > there were AFOS products entering the queue they would also show up
> > > > > > in one of the
> > > > > > three other LDM machines here.
> > > > > >
> > > > > > Actually, that may be a key, the buffer is expanding. Robb, do you
> > > > > > know how the
> > > > > > buffer plays into the LDM? Does LDM load a product
> > > > > > into a new dynamic buffer prior to placing it in the queue and then
> > > > > > does it free
> > > > > > the memory?\
> > > > > >
> > > > > > Thanks for your timely responses.
> > > > > >
> > > > > > Gregg
> > > > > >
> > > > > >
> > > > > > Gilbert Sebenste wrote:
> > > > > >
> > > > > > > On Thu, 4 Nov 1999, Robb Kambic wrote:
> > > > > > >
> > > > > > > > Gregory,
> > > > > > > >
> > > > > > > > After looking at the log messages, it doesn't appear your queue
> > > > > > > > is 250
> > > > > > > > megabytes. Look in the data dir, what's the:
> > > > > > > >
> > > > > > > > % ls -l ldm.pq
> > > > > > > >
> > > > > > > > If it's 250 megabytes then up it again. There might be other
> > > > > > > > products
> > > > > > > > coming in on your afos stream, etc. If it isn't then change it
> > > > > > > > in the
> > > > > > > > bin/ldmadmin file to 250. Then the standard
> > > > > > > >
> > > > > > > > % ldmadmin stop
> > > > > > > > % ldmadmin delqueue
> > > > > > > > % ldmadmin mkqueue
> > > > > > > > % ldmadmin start
> > > > > > > >
> > > > > > > > I just put the commands here for my sanity sake.
> > > > > > >
> > > > > > > DUH! I forgot he was also ingesting NOAAPORT. Whoa, that's 175 MB
> > > > > > > right
> > > > > > > there. Up, up and away! Use 400 MB at least.
> > > > > > >
> > > > > > > > Also, if you comment out the dds and pps does it make any
> > > > > > > > difference?
> > > > > > > >
> > > > > > > > Robb...
> > > > > > >
> > > > > > > I think that's the key here. NOAAPORT takes up 150 MB of my
> > > > > > > queue/hour,
> > > > > > > and of course, we filter out scrambled NIDS and other stuff. Give
> > > > > > > it a go,
> > > > > > > and then let us know!
> > > > > > >
> > > > > > > *******************************************************************************
> > > > > > > Gilbert Sebenste
> > > > > > > ********
> > > > > > > Internet: address@hidden (My opinions only!)
> > > > > > > ******
> > > > > > > Staff Meteorologist, Northern Illinois University
> > > > > > > ****
> > > > > > > Work phone: 815-753-5492
> > > > > > > ***
> > > > > > > *******************************************************************************
> > > > > > >
> > > > > >
> > > > >
> > > > > ===============================================================================
> > > > > Robb Kambic Unidata Program Center
> > > > > Software Engineer III Univ. Corp for Atmospheric
> > > > > Research
> > > > > address@hidden WWW: http://www.unidata.ucar.edu/
> > > > > ===============================================================================
> > > >
> > >
> > > ===============================================================================
> > > Robb Kambic Unidata Program Center
> > > Software Engineer III Univ. Corp for Atmospheric
> > > Research
> > > address@hidden WWW: http://www.unidata.ucar.edu/
> > > ===============================================================================
> >
>
> ===============================================================================
> Robb Kambic Unidata Program Center
> Software Engineer III Univ. Corp for Atmospheric Research
> address@hidden WWW: http://www.unidata.ucar.edu/
> ===============================================================================
>
> ------------------------------------------------------------------------
> Name: pqing.diff
> pqing.diff Type: Plain Text (TEXT/PLAIN)
> Encoding: BASE64
> Description: pqing.diff
>
> Name: pqing.c
> pqing.c Type: Plain Text (TEXT/PLAIN)
> Encoding: BASE64
> Description: pqing.c 1.66
begin:vcard
n:Grosshans;Gregory
tel;fax:405-579-0700
tel;work:405-579-0720
x-mozilla-html:TRUE
url:www.spc.noaa.gov
org:DOC/NOAA/NWS/NCEP/Storm Prediction Center
adr:;;1313 Halley Circle ;Norman;OK;73069;
version:2.1
email;internet:address@hidden
x-mozilla-cpt:;0
fn:Gregory Grosshans
end:vcard
