[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20000921: new mcidas routing problems



>From: "Neil R. Smith" <address@hidden>
>Organization: Texas A&M
>Keywords: 200009142317.e8ENHkb02698 ldm-mcidas 7.6.4 build

Neil,

>I'm having the dickens of a time trying to ferret out the production
>of gifs for our website that now are not being generated with the
>new 7.6.4 ldm-mcidas binaries.  This is a configuration that I
>inherited and have been able to avoid until now with the ldm-mcidas
>upgrade.  I've added the following pcact.conf entry:
>
>MCIDAS ^pnga2area Q. (..) (.*) (.*) (.*) (.*) (........) (....)
>       PIPE    -close
>       pnga2area -v
>       -a /data/mcidas/SATANNOT -b /data/mcidas/SATBAND
>       -l /unidata/ldm/logs/mcidas.log
>       -d /data/mcidas -r \1,\2

This looks correct.

>with the supplied new ROUTE.SYS installed in /data/mcidas.

I see the problem right here.  The copy of ROUTE.SYS distributed with
ldm-mcidas has the various PostProcessing entries turned off (SUSpended
in McIDAS parlance):

ROUTE LIST
S Pd         Description         Range       Last      Received  Post Process C
- -- ------------------------- --------- ------------ ---------- ------------ -
  CA Cloud Top Pressure        1100-1109     none        none    CTP.BAT      3
  CB Precipitable Water        1110-1119     none        none    PW.BAT       3
  CC Sea Sfc. Temperature      1120-1129     none        none    SST.BAT      3
  CD Lifted Index              1130-1139     none        none    LI.BAT       3
  CE CAPE                      1140-1149     none        none    CAPE.BAT     3
  CF Ozone                     1150-1159     none        none    OZONE.BAT    3
s CI GOES-E/W IR Composite       80-89       none        none        none     3
s CV GOES-E/W VIS Composite      90-99       none        none        none     3
s CW GOES-E/W H2O Composite      70-79       none        none        none     3
  LD NLDN Lightning Flashes      71-71       none        none        none     3
  M1 WETNET TEST IMAGE         1400-1409     none        none        none     3
s MA Surface MD data            default      none        none    SFC.BAT      3
s N1 GOES-East IR/TOPO Composi  220-229      none        none        none     3
s N2 GOES-East VIS/TOPO Compos  230-239      none        none        none     3
s N3 GOES-West IR/TOPO Composi  240-249      none        none        none     3
s N4 GOES-West VIS/TOPO Compos  250-259      none        none        none     3
s N5 MDR/TOPO Composite         260-269      none        none        none     3
s N6 Mollweide IR/TOPO Composi  270-279      none        none        none     3
s N7 GOES-E/W IR/TOPO Composit  280-289      none        none        none     3
s N8 GOES-E/W VIS/TOPO Composi  290-299      none        none        none     3
  NF Global Initialization Gri  101-102      none        none    GLOBAL.BAT   3
  NG Early Domestic Products      1-2        none        none    ADDGRID.BAT  3
  R1 Base Reflectivity Tilt 1   300-339      none        none        none     3
  R2 Base Reflectivity Tilt 2   340-379      none        none        none     3
  R3 Base Reflectivity Tilt 3   380-419      none        none        none     3
  R4 Base Reflectivity Tilt 4   420-459      none        none        none     3
  R5 Composite Reflectivity     460-499      none        none        none     3
  R6 Layer Reflect SFC-24 K ft  500-539      none        none        none     3
  R7 Layer Reflect 24-33 K ft   540-579      none        none        none     3
  R8 Layer Reflect 33-60 K ft   580-619      none        none        none     3
  R9 Echo Tops                  620-659      none        none        none     3
  RA Vertical Liquid H2O        660-699      none        none        none     3
  RB 1-hour Surface Rain Total  700-739      none        none        none     3
  RC 3-hour Surface Rain Total  740-779      none        none        none     3
  RD Storm Total Rainfall       780-819      none        none        none     3
  RE Radial Velocity Tilt 1     820-859      none        none        none     3
  RF Radial Velocity Tilt 2     860-899      none        none        none     3
  RG Radial Velocity Tilt 3     900-939      none        none        none     3
  RH Radial Velocity Tilt 4     940-979      none        none        none     3
  RI 248 nm Base Reflectivity   980-1019     none        none        none     3
  RJ Storm-Rel Mean Vel Tilt 1 1020-1059     none        none        none     3
  RK Storm-Rel Mean Vel Tilt 2 1060-1099     none        none        none     3
s RM Mandatory Upper Air MD da  default      none        none    MAN.BAT      3
s RS Significant Upper Air MD   default      none        none    SIG.BAT      3
  U1 Antarctic IR Composite     190-199      none        none        none     3
  U2 FSL2 hourly wind profiler  default      none        none        none     3
  U3 Manually Digitized Radar   200-209      none        none    MDR.BAT      3
s U4 Unidata-Wisconsin hourly   default      none        none    PROFILER.BAT 3
  U5 GOES-West US IR Band 4     130-139      none        none    GW-IR.BAT    3
  U6 FSL2 6-minute Wind profil  default  MDXX0099     00208 2137     none     3
  U9 GOES-West US Visible       120-129      none        none    GW-VIS.BAT   3
  UA Educational Floater I      160-169      none        none        none     3
  UB GOES-West US Water Vapor   170-179      none        none    GW-WV.BAT    3
  UC Educational Floater II      60-69       none        none        none     3
  UI GOES-East US IR Band 4     150-159      none        none    GE-IR.BAT    3
  UM Administrative Message     default      none        none        none     1
  UR Research Floater           180-189      none        none        none     3
  US Undecoded SAO Data         default      none        none        none     1
  UV GOES-East US Visible       140-149      none        none    GE-VIS.BAT   3
  UW GOES-East US Water Vapor   210-219      none        none    GE-WV.BAT    3
  UX Mollweide Composite IR     100-109      none        none    MOLL.BAT     3
  UY Mollweide Composite H2O    110-119      none        none        none     3
ROUTE: Done

The entries you are probably concerned with are CI, CV, CW, and N1 - N8.
These can be turned on from a McIDAS-X session as follows:

<login as 'mcidas'>
cd workdata
<start a McIDAS-X session>
<verify that you have a REDIRECTion that points to the copy of ROUTE.SYS
in /data/mcidas:

REDIRECT ADD ROUTE.SYS "/data/mcidas

<turn on the PostProcessing actions>

ROUTE REL UI UV UW
ROUTE REL N1 N2 N3 N4 N5 N6 N7 N8

Once the actions are activated (not turned off), the PostProcessing
should proceed IF the Bourne shell script 'batch.k' that is being
run by pnga2area has been properly configured (edited to set needed
McIDAS environment variables).

Also, the names of the McIDAS BATCH files that are kicked off by the
ldm-mcidas version of ROUTE.SYS were changed to more correctly identify
the satellite platform:

   old BATCH file name        new BATCH file name
   ----------------------+-----------------------
        IR8.BAT                   GE-IR.BAT
        VIS8.BAT                  GE-VIS.BAT
        H2O8.BAT                  GE-WV.BAT
        IR9.BAT                   GW-IR.BAT
        VIS9.BAT                  GW-VIS.BAT
        H2O9.BAT                  GW-WV.BAT

>SATANNOT and SATBAND are the supplied new items.

OK.  I would have put these in ~ldm/etc, but this has nothing to do
with the PostProcesses running.

>This effort produced none of the old gifs, so I copied back in 
>the old ROUTE.SYS, and now I get gifs with black (empty) graphic
>content. 

Hmm...  I can't explain this one unless it is an indication that
you are decoding the CIMSS products using the routing table and
your old routing table does not have CIMSS product entries.

>For the old ldm-mcidas, I've been able to find a DROUTE.BAT that
>looks like the one to generate our old ROUTE.SYS.

Right.

>So I see that ROUTE.SYS picks the products to ROUTE and RELease.

Yes, exactly.

>Now
>the old pqact.conf config had only lwfile, gunrv2, and lwtoa3 
>entries writing to /data/mcidas, but our web product gifs were
>being written to our web server document directory structure.

OK.  These pqact.conf actions are now archaic and can be removed.

>I find an LWPATH.NAM that has the correct redirects for the exact 
>gifs I'm missing, and *.BAT (eg. IR8.BAT) files that have local
>mods generating the exact gif filenames I need.

OK, here is another place where the ldm-mcidas routing table entries
don't match your system.  The new routing table is trying to run
GE-IR.BAT and you have a file called IR8.BAT that you want to use.
You can either change the routing table entry to run IR8.BAT or
you can rename your BATCH file to GE-IR.BAT.

Hopefully, you have moved the BATCH files you want to use to the 
~mcidas/workdata directory and not left them in ~mcidas/data.  The
reason for this is that the old BATCH files will be overwritten by
new McIDAS-X installations if they are left in ~mcidas/data.  The
other option is to rename the BATCH files, but that then requires
you to change your routing table entries that kick off PostProcessing.

If you decide to use the ldm-mcidas routing table, you should affect
the name change while moving the BATCH files from ~mcidas/data to
~/mcidas/workdata.

>But how do I get these things to play ball with the new pnga2area 
>products?

They should play automatically IF the routing table entries are
setup correctly (i.e., not SUSpended and reference BATCH files
that exist).

>Can someone please give me an example production trail of, say, 
>a gif for GOES-8 IR centered on the central Gulf of Mexico, from
>ldm ingest to custom destination directory for the new binary set?

Here goes:

o the PNG compressed GOES-East (GOES-8 right now) image is sent to
  your LDM machine.  It will have a header that looks like:

  pnga2area Q1 UI 151 GOES-8_IMG 10.7um 4km 20000921 1715

o your LDM's pqact.conf file will have an entry that is what you
  sent above:

MCIDAS ^pnga2area Q. (..) (.*) (.*) (.*) (.*) (........) (....)
       PIPE    -close
       pnga2area -v
       -a /data/mcidas/SATANNOT -b /data/mcidas/SATBAND
       -l /unidata/ldm/logs/mcidas.log
       -d /data/mcidas -r \1,\2

o pnga2area needs/expects ROUTE.SYS to exist in /data/mcidas AND it must
  be readable and writable by the user running the LDM

o ROUTE.SYS should have an entry for product UI that looks like:

  UI GOES-East US IR Band 4     150-159      none        none    GE-IR.BAT    3

o after the GOES-East IR product is correctly decoded, pnga2area attempts
  to run the command:

  batch.k UI n yyddd hhmmss cc "GE-IR.BAT
          |  |   |     |    |     |_______ BATCH file name
          |  |   |     |    |______________ ingest condition code
          |  |   |     |___________________ TIME (HHMMSS)
          |  |   |_________________________ DAY (YYDDD)
          |  |_____________________________ AREA file number
          |________________________________ Product code

o you should have setup your LDM account to have a ~ldm/decoders directory
  in which you have copied the Bourne shell script 'batch.k' (distributed
  with ldm-mcidas; you must already have an existing copy that is
  configured correctly).  ~ldm/decoders/batch.k must be executable and
  ~ldm/decoders must exist in the PATH of the user running the LDM.
  NOTE: you should NOT have _any_ McIDAS environemnt "stuff" defined
  in the LDM environment: no environment variables; no PATH entry for
  ~mcidas/bin; and no LD_LIBRARY_PATH entry for ~mcidas/lib!

o the Bourne shell script, assumed to be configured with necessary
  McIDAS environment variable information, is the thing that is run
  by pnga2area.  'batch.k' will, after setting up needed McIDAS environment
  variable "stuff", run the "real" McIDAS BATCH command (~mcidas/bin/batch.k).

o the "real" McIDAS BATCH command will run the named BATCH file IF it exists
  and can be found in the MCPATH set in 'batch.k' script, or if there is
  a McIDAS REDIRECTion, contained in ~mcidas/workdata/LWPATH.NAM, that
  locates the BATCH file

>Much thanks, -Neil

I think that after you RELease the entries in the ldm-mcidas version of
ROUTE.SYS and change the names of the McIDAS BATCH files you are using
to match the ROUTE.SYS entries, your PostProcessing will once again be
operational.  Please let me know if this is not the case.

>-- 
>Neil R. Smith, Comp. Sys. Mngr.                address@hidden
>Dept. Atmospheric Sci., Texas A&M Univ.        979/845-6272 FAX:979/862-4466

Tom Yoksas