[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20040409: Vietnam and GEMPAK GRIB decoding (cont.)

>From: Mai Nguyen <address@hidden>
>Organization: National Center for Hydro-Meteorological Forecasting of Vietnam
>Keywords: 200312020023.hB20N4p2027742 IDD LDM Linux GEMPAK

Mai,

re: making appropriate runtime links for starting/stopping LDM under/etc/rc?.d
>I think i've done that. Please have a look.

Looks good.

re: injecting GRIB messages into LDM queue so they will be decoded by
GEMPAK's dcgrib2
>Could you please tell me how to put them in ldm queu.
>Is it to change the qpact.gempak?

This will have to wait for another email.

>I have a good news that we were able to decode (using
>dcgrib2) and display (on NAWIPS) our model data (we
>currently have 3 models for local weather) manually.
>So please discard my prblem of GRIB decoding. But now,
>how do I put it in the scripts for LDM to do it
>automatically?
>
>So, given those problems solved, we will still have
>the problems of ingesting our observation data and
>transforming our satellite images. We hope to do it
>ASAP.

Chiz and I took a look at your surface observation file,
~ldm/VNdata/SYNOP/AAXX00 and don't know what some of the information
is.  We can see that a good portion of each report follows the WMO
standard, FM-12, for synoptic reports.  Chiz developed a "technique" to
strip out the portions of the obs that don't follow FM-12 and then
decode the resulting set using the GEMPAK land surface decoder dclsfc.
Here is a quick rundown on the procedure, all done as the user 'ldm':

1) the file in ~ldm/VNdata is in DOS format (was it created on
   a Windows machine?).  The first thing that is needed is to
   convert it to Unix format:

   cp ~ldm/VNdata/SYNOP/AAXX00 /tmp
   dos2unix AAXX00

   This converts the file 'in-place': AAXX00 is overwritten as
   a Unix file.

2) there is more than one observation per line.  We used a sed
   script to convert multiple obs per line to one ob per line:

cat AAXX00 | sed 's/=  */=\n/g' > AAXX00.1

3) here are examples of report segments we don' understand:

722030024004AAXX48/8622000022001M048/86 ...
402020028004AAXX4800402000002004K048004 ...
422030010004AAXX4800822000022004K048008 ...
614030056004AAXX4801014000014004K048010 ...
 etc.

cat AAXX00.1 | tr -d '\000' > AAXX00.2
cat AAXX00.2 | sed 's/^.*AAXX..................//' > AAXX00.3

   The first line (cat AAXX00.1) gets rid of bogus (nil) characters.
   The second strips out the parts we don't understand.

   Also, some of the apparent station IDs are suspect.  Here
   are a couple of examples:

   48/89
   48/93

   The station ID can not have '/' in it.

4) next, add appropriate headers to make the file contents look
   like a WMO bulletin:

^A^M^M
999
AAXX 22004

 ...                  <- stripped file contents from steps above

^M^M
^C

   NOTES:  '^A^M^M'  -> CTRL-A CTRL-M CTRL-M (SOH \r \r)
           '999'     -> broadcast sequence number
           'AAXX 22004' -> AAXX type followed by DDHHn
                           DD -> day
                           HH -> hour
                           n  -> type of unit (see info for FM-12 from WMO)
           '^M^M'       -> CTRL-M CTRL-M (\r \r)
           '^C'         -> CTRL-C (ETX)

5) source the Gemenviron file so GEMPAK environment variables are defined:

source ~gempak/NAWIPS/Gemenviron

6) run the GEMPAK dclsfc decoder to create the output GEMPAK file:

cat AAXX00.3 | dclsfc -v 4 -d - -c 040322/0000 YYYYMMDD_syn.gem

   NOTES:  '-v 4' -> very verbose (level 4) logging
           '-d -' -> log to stdout
           '-c 040322/0000' == 'c yymmdd/hhmm'  -> specify year, month,
                                                   day, hour, minute of data
           'YYYYMMDD_syn.gem' -> pattern for name of output (decoded) file

   You can look at the help output from dclsfc ('dclsfc -help') to
   learn more about flags that dclsfc understands/uses.
     

The files created in the sequence listed above can be found in /tmp
on met_research3:

AAXX00   -> copy of original file converted from DOS to Unix format
AAXX00.1 -> added line breaks between reports
AAXX00.2 -> strip out bogus (nil) characters
AAXX00.3 -> strip out portions of reports that do not follow FM-12
20040322_syn.gem -> resultant file of decoded obs

>Thanks Tom, and have a nice weekend.

You too.

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publically available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.