Re: 20021126: converting old wmo metar data to gdbm?

• To: page.christian@xxxxxxx
• Subject: Re: 20021126: converting old wmo metar data to gdbm?
• From: Robb Kambic <rkambic@xxxxxxxxxxxxxxxx>
• Date: Wed, 27 Nov 2002 09:33:16 -0700 (MST)

>To: support@xxxxxxxxxxxxxxxx
>From: =?ISO-8859-1?Q?Christian_Pag=E9?= <page.christian@xxxxxxx>
>Subject: converting old wmo metar data to gdbm?
>Organization: UCAR/Unidata
>Keywords: 200211011624.gA1GO6X00936

Hi,

Is there a way to convert old metar archived data from raw wmo format to gdbm format?

In realtime, it uses pqsurf with a DBFILE action.

Christian,

You are asking a tough question.  It can be done with some work though.  I
read your support question from July 31 about a similiar problem.  Here's
some background info that really important.  For pqsurf to work off a ldm

any actions on them.  What this means is that you can't take an hours
worth of bulletin and pqinsert into an ldm queue because they would have

Solution:

You have to break up the hourly files into the original bulletins, ie

^A^M^M
981 ^M^M
SAFR31 LFPW 311230^M^M
LFJL 311230Z AUTO 26010KT 9999 SCT058 SCT070 30/15 Q1022=^M^M
LFBG 311230Z 36006KT 320V040 CAVOK 31/16 Q1024=^M^M
^M^M
^C

Using pqinsert, send it to the ldm with name "SAFR31 LFPW 311230"

% pqinsert -vl - -s 999 -f IDS -p "SAFR31 LFPW 311230" <rawbulletinFile>

where <rawbulletinFile> contains:

^A^M^M
981 ^M^M
SAFR31 LFPW 311230^M^M
LFJL 311230Z AUTO 26010KT 9999 SCT058 SCT070 30/15 Q1022=^M^M
LFBG 311230Z 36006KT 320V040 CAVOK 31/16 Q1024=^M^M
^M^M
^C

Code:


One could make a perl script to do the work. Something like this:
Script name bulletinInsert

#!/local/bin/perl
#
# Now begin parsing file and decoding observations breaking on cntrl C
$/ = "\cC" ;

while( <STDIN> ) {
        # extract product ID
        /(\w{4}\d{2} \w{4} \d{6}})/s
        $prodID = $1 ;
        open( OUT,">raw" ) ;
        pring OUT $_ ;
        close OUT ;
        `pqinsert  -vl - -s 999 -f IDS -p "$prodID" raw` ;
        unlink "raw" ;
}

To run:

% bulletinInsert < <raw hourly files>

I would extract one bullletin and test pqinsert first, then look at the
queue using pqcat to make sure the prodID is correct. Then use
bulletinInsert and check queue again with pqcat.  I didn't debug the
script, you might want to run it in debug mode to check it.

% perl -d bulletinInsert < <raw hourly files>

Hopes this works for you,
Robb...

Thanks,

Christian Pagé
page@xxxxxxxxxxx
http://meteocentre.com/    http://meteoalerte.com/

Etudiant au Doctorat en Sciences de l'environnement UQAM
+1 514 987 3000 ext. 2376


------- End of Forwarded Message

==============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
rkambic@xxxxxxxxxxxxxxxx                   WWW: http://www.unidata.ucar.edu/
==============================================================================