[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: compare desi to tg2 (fwd)

Chad,

desi, our NOAAport ingestor is Solaris 2.7

Sun Microsystems Inc.   SunOS 5.7       Generic October 1998
# uname -a
SunOS desi.unidata.ucar.edu 5.7 Generic_106542-04 i86pc i386 i86pc

We are still getting the bit error messages, inge.log

NMC 1999.173.194735:bit error
NMC 1999.173.195746:bit error
NMC 1999.173.200252:bit error
NMC 1999.173.204004:hdlc problem
NMC 1999.173.204422:bit error
NMC 1999.173.204456:bit error
NMC 1999.173.204832:bit error
NMC 1999.173.205340:bit error
NMC 1999.173.205340:bit error
NMC 1999.173.205340:bit error
NMC 1999.173.205340:bit error
NMC 1999.173.205340:bit error
NMC 1999.173.210901:bit error
NMC 1999.173.211820:bit error
NMC 1999.173.211827:bit error
NMC 1999.173.211917:bit error  

Robb...

On Tue, 22 Jun 1999, Chad Johnson wrote:

> Robb Kambic wrote:
> > 
> > Chad,
> > 
> > I created a ldm release ldm-5.0.8 that appends the PIL to the WMO header
> > so users can key on it. It's a modification to pqing, etc.  Can you
> > install this release on you noaaport machine?  I think Chiz wants to
> > announce the PIL today.
> 
> OK. I'll get it installed today. I'll let you know when I'm finished.
> 
> > Your ideas about comparing the 'bit error' messages is good one, but the
> > tg2 machine is no longer connected to the NOAAport feed. The tg2 machine
> > is the one that is getting prepared to ship to LSU. If the situation
> > arises again then I'll check the messages.
> 
> We think we found the culprit to our bit error problem and I wonder if it 
> might
> be related to the problems you are seeing. It appears as though there is a 
> bug in
> SolarisX86 2.5.1 when memory mapping files in a tmpfs file system which have 
> been
> modified, but not removed (which is what happens with the latest mod of 
> inge). To
> fix the problem we moved the creation of the spool files from /tmp (a tmpfs 
> file
> system) to /data (a ufs file system). Please check which OS version you are
> running on both your systems. We don't appear to be having this problem with
> SolarisX86 2.6 or 7. If you are running 2.5.1 with the latest inge mods and 
> the
> spool files are being placed in /tmp, we recommend that the spool files be 
> placed
> in /data. This can be modified by editing the file /etc/ingcntl. Under the
> 'start' case, change the line which reads 'cd /tmp' to 'cd /data'. Stop and
> restart the ingestor.
> 
> 
> -Chad
> 
> > 
> > Thanks for installing the new ldm release,
> > Robb...
> > 
> > On Thu, 10 Jun 1999, Chad Johnson wrote:
> > 
> > > Robb Kambic wrote:
> > > >
> > > > Chad,
> > > >
> > > > Here's some more info Chiz uncovered.  Actually this seems like a timing
> > > > error for inge, just a thought.
> > >
> > > The ingestor software writes error messages such as 'bit error', 'hdlc 
> > > problem',
> > > 'overhang', etc when it encounters a problem with the datastream. If one 
> > > of these
> > > problems occur when a product is in transmission, this product will be 
> > > dropped.
> > > Could you compare these error messages between the two systems and see if 
> > > they
> > > are identical? If any errors do not occur at the same time on both 
> > > systems, this
> > > may account for the differences you are seeing in the products reaching 
> > > the LDM.
> > >
> > > -Chad
> > >
> > > >
> > > > Robb,,,
> > > >
> > > > ===============================================================================
> > > > Robb Kambic                                Unidata Program Center
> > > > Software Engineer III                      Univ. Corp for Atmospheric 
> > > > Research
> > > > address@hidden                   WWW: http://www.unidata.ucar.edu/
> > > > ===============================================================================
> > > >
> > > > ---------- Forwarded message ----------
> > > > Date: Wed, 9 Jun 1999 12:53:48 -0600
> > > > From: Steve Chiswell <address@hidden>
> > > > To: address@hidden, address@hidden
> > > > Subject: compare desi to tg2
> > > >
> > > > This morning Mike reverted to the previous inge on tg2
> > > > for comparison against desi running SSEC's latest version.
> > > >
> > > > Here are some comparisons:
> > > >
> > > > from 1640Z through 1822Z, the files /scratch/chiz/tg2.log1
> > > > and /scratch/chiz/desi.log1 are sync'd to both start with
> > > > product 738 and end with product 982 for DDPLUS|IDS products.
> > > >
> > > > %wc -l tg2.log1 desi.log1
> > > >           7833 tg2.log1
> > > >           7804 desi.log1
> > > >
> > > > tg2 obtained 29 more DDPLUS products.
> > > >
> > > > Comparing both files with just the WMO header parts shows that
> > > > both machines saw some products the other one didn't. This
> > > > would seem to indicate that the latest version does not fix the previous
> > > > problem we were seeing with missing products.
> > > >
> > > > diff tg2.sav desi.sav
> > > > 122a123,126
> > > > > IDS|DDPLUS 214  FQUS23 KWBC 091638
> > > > > IDS|DDPLUS 215  SIVD80 KWBC 091500 RRD
> > > > > IDS|DDPLUS 216  SIVF30 KWBC 091500 RRB
> > > > > IDS|DDPLUS 217  SPCN43 CWAO 091638
> > > > 563,565d566
> > > > < IDS|DDPLUS 891  WSCN33 CWEG 091646
> > > > < IDS|DDPLUS 892  WWIN40 DEMS 091200
> > > > < IDS|DDPLUS 893  WSCN34 CWEG 091647
> > > > 1092d1092
> > > > < IDS|DDPLUS 708  WWCN31 CWXE 091657
> > > > 2179,2194d2178
> > > > < IDS|DDPLUS 941  SAAK31 KWBC 091600 RRE /pMETAR
> > > > < IDS|DDPLUS 942  SACA42 KWBC 091700 /pMETAR
> > > > < IDS|DDPLUS 943  SACA42 KWBC 091600 RRA /pMETAR
> > > > < IDS|DDPLUS 944  SAHW31 KWBC 091700 /pMETAR
> > > > < IDS|DDPLUS 945  SAPA41 KWBC 091700 /pMETAR
> > > > < IDS|DDPLUS 946  FEXW40 KWBC 091200
> > > > < IDS|DDPLUS 947  FEXW41 KWBC 091200
> > > > < IDS|DDPLUS 948  FEXW42 KWBC 091200
> > > > < IDS|DDPLUS 949  FEXW43 KWBC 091200
> > > > < IDS|DDPLUS 950  FEXW44 KWBC 091200
> > > > < IDS|DDPLUS 951  FEXW45 KWBC 091200
> > > > < IDS|DDPLUS 952  FEXW46 KWBC 091200
> > > > < IDS|DDPLUS 953  FEXW47 KWBC 091200
> > > > < IDS|DDPLUS 954  FEXW48 KWBC 091200
> > > > < IDS|DDPLUS 955  FEXW49 KWBC 091200
> > > > < IDS|DDPLUS 956  FEXW50 KWBC 091200
> > > > 3240a3225,3226
> > > > > IDS|DDPLUS 308  FTPA31 PHNL 091700
> > > > > IDS|DDPLUS 309  UBUS01 KNKA 091722
> > > > 3356,3367d3341
> > > > < IDS|DDPLUS 489  UAPA01 KWBC 091722
> > > > < IDS|DDPLUS 490  FXUS07 KBOI 091718 PAA /pFWFBOI
> > > > < IDS|DDPLUS 491  SPCN43 CWAO 091720 CCA
> > > > < IDS|DDPLUS 492  UACN10 CYBW 091722
> > > > < IDS|DDPLUS 493  FTAS31 VTBB 091600 RRB
> > > > < IDS|DDPLUS 504  SRUS53 KABR 091721 /pRR3ABR
> > > > < IDS|DDPLUS 510  SRUS55 KMSO 091721 /pRR1MSO
> > > > < IDS|DDPLUS 512  FTVN20 SVMI 091618
> > > > < IDS|DDPLUS 513  UAXX36 KAWN 091700 RRW
> > > > < IDS|DDPLUS 514  FPAK20 PANC 091722
> > > > < IDS|DDPLUS 515  FACN33 CWUL 091730
> > > > < IDS|DDPLUS 516  FPUS74 KCRP 091722 /pNOWCRP
> > > > 5209a5184,5188
> > > > > IDS|DDPLUS 842  CXUS33 KOAX 091752 /pSCDOAX
> > > > > IDS|DDPLUS 843  SRUS76 KHNX 091750 /pRR7FAT
> > > > > IDS|DDPLUS 845  SRUS73 KFSD 091750 /pRR7SUX
> > > > > IDS|DDPLUS 848  SXUS01 KNYC 091753 /pOSONYC
> > > > > IDS|DDPLUS 849  SXUS42 KTAE 091755 /pOSOTLH
> > > > 5429d5407
> > > > < IDS|DDPLUS 089  SXUS70 KWAL 091756
> > > > 5913,5919d5890
> > > > < IDS|DDPLUS 646  SRUS71 KILN 091800 /pRR7CVG
> > > > < IDS|DDPLUS 647  SRUS71 KGYX 091800 /pRR7AKQ
> > > > < IDS|DDPLUS 648  SRUS73 KLOT 091800 /pRR7P75
> > > > < IDS|DDPLUS 649  SRUS76 KMTR 091800 /pRR7SCK
> > > > < IDS|DDPLUS 651  SRUS73 KGRR 091800 /pRR7GRR
> > > > < IDS|DDPLUS 652  SRUS72 KFFC 091800 /pRR7AHN
> > > > < IDS|DDPLUS 653  SRUS72 KFFC 091800 /pRR7ATL
> > > >
> > > > files tg2.log2 and desi.log2 contain HDS products from
> > > > 732 through 905 from  1639Z to 1830Z.
> > > >
> > > > %wc -l tg2.log2 desi.log2
> > > >           4600 tg2.log2
> > > >           4609 desi.log2
> > > >
> > > > desi obtained 9 more hds products.
> > > >
> > > > Diffing the WMO header portion of these files shows again that both
> > > > ingesters missed different products. Note the difference between:
> > > > < HDS 201  HUJK20 KWBC 091200 /m93
> > > > ---
> > > > > HDS 201  HUJK20 KWBC 091200 /mAVN
> > > >
> > > > This difference in the WMO header is cause by a difference in the bits
> > > > that pqing uses to identify the model number in the PDS block of the 
> > > > grib
> > > > product. This would rule out a satellite dish error since both machines
> > > > see the same data stream feed. The difference in the products
> > > > would therefore have to be originating in the spool files.
> > > >
> > > > diff tg2.savhds desi.savhds
> > > > 513a514
> > > > > HDS 463  HUOJ25 KWBC 091200 /mAVN
> > > > 1114c1115
> > > > < HDS 201  HUJK20 KWBC 091200 /m93
> > > > ---
> > > > > HDS 201  HUJK20 KWBC 091200 /mAVN
> > > > 1485a1487
> > > > > HDS 630  HOMK85 KWBC 091200 /mAVN
> > > > 2287a2290
> > > > > HDS 909  YRQJ70 KWBC 091200 /mAVN
> > > > 3083a3087
> > > > > HDS 317  SDUS97 KTSA 091712
> > > > 3322a3327,3334
> > > > > HDS 310  SDUS99 KWBC 091720
> > > > > HDS 311  SDXX99 KWBC 091721
> > > > > HDS 312  SDUS99 KWBC 091723
> > > > > HDS 313  SDXX99 KWBC 091721
> > > > > HDS 314  SDXX99 KWBC 091722
> > > > > HDS 315  SDUS99 KWBC 091723
> > > > > HDS 316  SDXX99 KWBC 091722
> > > > > HDS 317  SDXX99 KWBC 091723
> > > > 3387,3392d3398
> > > > < HDS 503  SDUS42 KTAE 091723
> > > > < HDS 505  SDUS46 KSGX 091724
> > > > < HDS 506  SDXX99 KLKN 091722
> > > > < HDS 507  SDXX99 KWBC 091721
> > > > < HDS 508  SDXX99 KBYZ 091724
> > > > < HDS 509  SDUS99 KWBC 091723
> > > > 3445d3450
> > > > < HDS 943  SDUS97 KBIS 091730
> > > > 3480a3486
> > > > > HDS 276  SDUS97 KSGF 091730
> > > > 3503,3507d3508
> > > > < HDS 395  SDUS97 KDDC 091733
> > > > < HDS 396  SDUS97 KLWX 091726
> > > > < HDS 397  SDUS51 KLWX 091726
> > > > < HDS 398  SDUS97 KTSA 091735
> > > > < HDS 399  SDUS97 KBIS 091735
> > > > 3679a3681,3683
> > > > > HDS 283  SDUS97 KLBF 091733
> > > > > HDS 284  SDUS97 KAMA 091725
> > > > > HDS 285  SDUS97 KEAX 091740
> > > > 3865a3870,3876
> > > > > HDS 838  SDUS42 KTAE 091753
> > > > > HDS 839  SDUS46 KSGX 091754
> > > > > HDS 840  SDXX99 KWBC 091752
> > > > > HDS 841  SDXX99 KWBC 091751
> > > > > HDS 844  SDXX99 KLKN 091752
> > > > > HDS 846  SDXX99 KBYZ 091754
> > > > > HDS 847  SDUS99 KWBC 091753
> > > > 3958,3959d3968
> > > > < HDS 645  AGUS43 KUNR 091802
> > > > < HDS 650  AGUS46 KPQR 091801
> > > >
> > > > At this point, it does not appear that the Lone SOH messages in the
> > > > log files is significantly different between the two versions of inge.
> > > > Thelma is running up to date at present, but it will be interesting to
> > > > see if the latencies reoccur today.
> > > >
> > > > Chiz
> > >
> > > --
> > > Chad W. Johnson                           E-mail: address@hidden
> > > Programmer/Meteorologist                  Voice: (608) 265-5292
> > > Space Science and Engineering Center      Fax: (608) 263-6738
> > > University of Wisconsin -- Madison
> > >
> > 
> > ===============================================================================
> > Robb Kambic                                Unidata Program Center
> > Software Engineer III                      Univ. Corp for Atmospheric 
> > Research
> > address@hidden                   WWW: http://www.unidata.ucar.edu/
> > ===============================================================================
> 
> -- 
> Chad W. Johnson                           E-mail: address@hidden
> Programmer/Meteorologist                  Voice: (608) 265-5292
> Space Science and Engineering Center      Fax: (608) 263-6738
> University of Wisconsin -- Madison
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================