[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: SAUS4* headers not being processed (fwd)

On Mon, 7 Aug 2000, Tom McDermott wrote:

> On Mon, 7 Aug 2000, Robb Kambic wrote:
> 
> > At this time, I might have a handle on the pqsurf problems.  As Dan Vietor
> > noted that the products are inconsistently inserted into the NOAAport
> > stream using a variations of patterns in the bulletin headings and report
> > headings.  One problem was the bulletins where initially designated as
> > METARs but actually reports were SPECIs.  The surf_split.c code was
> > enhanced to catch these bulletins. There is a modified surf_split.c in
> > the Unidata LDM ftp directory.  To upgrade you LDM, copy the surf_split.c
> > code into the src/pqsurf directory and then perform make in the pqsurf
> > directory. The final step is to copy pqsurf to ../bin .  The new
> > surf_split code is backward compatiable only to ldm-5.1.1 release. 
> > 
> > The following 3 pqsurf entries are writing to the same output file, this
> > causes file corruption and missed data.  I created one pqsurf entry that
> > encompases the three after them.
> > 
> > > # Dumping both SAOs and METARs into one file.
> > > # SAOs.
> > > WMO       ^sao (.*) (.*) ([0-3][0-9])([0-2][0-9])
> > >   FILE    domestic/surface_obs/(\3:yy)(\3:mmm)\3_\4
> > > # METARs.
> > > WMO       ^metar (.*) ([0-3][0-9])([0-2][0-9])
> > >   FILE    domestic/surface_obs/(\2:yy)(\2:mmm)\2_\3
> > > # METARs.
> > > WMO       ^speci (....) ([0-3][0-9])([0-2][0-9])
> > >   FILE    domestic/surface_obs/(\2:yy)(\2:mmm)\2_\3
> > > 
> > 
> > WMO   ^(sao .. ...|metar ....|speci ....) ([0-3][0-9])([0-2][0-9])
> >        FILE    domestic/surface_obs/(\2:yy)(\2:mmm)\2_\3
>  
> Robb,
> 
> In accordance with these instructions, I changed our 'pqsurf.conf' from:
> 
> #
> #  create products for weather program
> #
> WMO   ^sao .. (...) ([0-3][0-9])([0-2][0-9])
>       DBFILE
> /weather/ldm/data/weather/(\2:yyyy)(\2:mm)\2/(\2:yyyy)(\2:mm)\2\3.SA_DB \1
> WMO   ^sao ... (...) ([0-3][0-9])([0-2][0-9])
>       DBFILE
> /weather/ldm/data/weather/(\2:yyyy)(\2:mm)\2/(\2:yyyy)(\2:mm)\2\3.SA_DB \1
> #
> WMO   ^metar (....) ([0-3][0-9])([0-2][0-9])
>       DBFILE
> /weather/ldm/data/weather/(\2:yyyy)(\2:mm)\2/(\2:yyyy)(\2:mm)\2\3.metar_gdb
> \1
> WMO   ^speci (....) ([0-3][0-9])([0-2][0-9])
>       DBFILE
> /weather/ldm/data/weather/(\2:yyyy)(\2:mm)\2/(\2:yyyy)(\2:mm)\2\3.metar_gdb
> \1
> 
> 
> to:
> 
> 
> #
> WMO   ^sao (.. ...|... ...) ([0-3][0-9])([0-2][0-9])
>       DBFILE
> /weather/ldm/data/weather/(\2:yyyy)(\2:mm)\2/(\2:yyyy)(\2:mm)\2\3.SA_DB \1
> #
> WMO   ^(metar ....|speci ....) ([0-3][0-9])([0-2][0-9])
>       DBFILE
> /weather/ldm/data/weather/(\2:yyyy)(\2:mm)\2/(\2:yyyy)(\2:mm)\2\3.metar_gdb
> \1
> 

Tom,

The \1 refers to the first set of parens, \2 refers to the second set, etc 
in the pattern match.  The original entry \1 didn't contain any spaces.
Maybe that's the problem. The reason the \1 follows at the end of the
entry line is that it's the key that GDBM uses to insert the report into
the file.  Here's how I would rewrite the entries:


WMO   ^sao ...? (...) ([0-3][0-9])([0-2][0-9])
       DBFILE
 /weather/ldm/data/weather/(\2:yyyy)(\2:mm)\2/(\2:yyyy)(\2:mm)\2\3.SA_DB 
 \1

added the ?, in RE it means zero or one occurance.


WMO   ^(metar|speci) (....) ([0-3][0-9])([0-2][0-9])
       DBFILE

/weather/ldm/data/weather/(\3:yyyy)(\3:mm)\3/(\3:yyyy)(\3:mm)\3\4.metar_gdb
 \2

Added another set of parens so I increased all the numbers in the entry by
one.


Robb...


> After HUPing the pqact for pqsurf, this resulted in that pqact utilizing
> >40% of CPU, and created an hourly file of 42MB within a few minutes (as
> opposed to a normal file of 700K to 800K).  So I think I did something
> wrong, but not sure what.
> 
> Also, I have no idea what the '\1' reference is doing there, with a space
> between the output file name and the reference, but I think it was copied
> from the 'weather' program README.
> 
> Tom
> ------------------------------------------------------------------------------
> Tom McDermott                         Email: address@hidden
> System Administrator                  Phone: (716) 395-5718
> Earth Sciences Dept.                  Fax: (716) 395-2416
> SUNY College at Brockport
> 
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================