[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20000707: TOPO/Composite creation by ROUTE PostProcess BATCH invocations (cont.)



>From: "James D. Marco" <address@hidden>
>Organization: Cornell
>Keywords: 200007062021.e66KLJT26042 McIDAS-X ROUTE PP BATCH

Jdm,

OK, so I said I would be poking around all afternoon, but I decided to
respond early (and often :-).

>       The primary LDM/McIDAS machine:
>                       name: vorticity.cit.cornell.edu (DNS registered)
>                       IP:   132.236.186.25
>                       OS:   RedHat 6.2
>                       Arch: Dual PentIII 550, 256M Ram, 18g UW-SCSI HD
>       Vorticity feeds LDM to two other machines 
>               Snowstorm.cit.cornell.edu:  ETA grids for MM5 initializations
>                                                for Gempack
>               Visibility.cit.cornell.edu: for DIFAX (setting this up soon)
>                 for McIDAS  
>
>       Vorticity also exports (nfs) the decoded data to 5 other machines
>used for McIDAS display servers. Most of the NT machines can run 'exceed' 
>to view 'canned' McIDAS batch displays from there. (BTW: Did you know that 
>the McIDAS GUI will not run on NT/WindowsXX over the Exceed emulator?)

No, I did not know that.  At some point in the past I did run it under
Windows 95 and Exceed, but I havn't tried it in a long time.  We can
deal with this later.

>Lets see...OK. from vorticity's LDM account, for batch.k I get:        
>       
>               ldm@vorticity> which batch.k
>               /usr/local/ldm/decoders/batch.k 
>       This is the script wrapper for the actual call to:
>               ~mcidas/bin/batch.k
>               The pathing appears to be correct.   
>The ROUTEPP.LOG looks like this (not good):  
>       ldm@vorticity:/home/mcidas/data> cat ROUTEPP.LOG
>       BATCH UX 102 189 13057 1 TWIN=0 "MOLL.BAT
>       BATCH UX 103 189 43009 1 TWIN=0 "MOLL.BAT
>       BATCH UX 104 189 73020 1 TWIN=0 "MOLL.BAT

This is telling us that the only product that has a ROUTE PostProcess
BATCH invocation defined for it is UX, the Mollweide Composite IR image.

After logging in as mcidas, I did the following to verify this:

cd data
dmap.k ROUTE.SYSPERM      SIZE LAST CHANGED FILENAME  DIRECTORY
---- --------- ------------ --------- ---------
-rw-     16128 Jul 07 13:42 ROUTE.SYS /var/data/mcidas
16128 bytes in 1 files

Good, there is a copy of ROUTE.SYS in the directory in which the images
will be decoded by the ldm-mcidas decoder lwtoa3.

ls -l /var/data/mcidas/ROUTE.SYS 
-rw-rw-r--    1 mcidas   unidata     16128 Jul  7 13:42 
/var/data/mcidas/ROUTE.SYS

Also good, it is readable and writable by both the ldm and mcidas.

route.k LIST
S Pd         Description         Range       Last      Received  Post Process C
- -- ------------------------- --------- ------------ ---------- ------------ -
  CA Cloud Top Pressure        1100-1109 AREA1104     00189 1715     none     3
  CB Precipitable Water        1110-1119 AREA1113     00189 1632     none     3
  CC Sea Surface Temperature   1120-1129 AREA1127     00189 1532     none     3
  CD Lifted Index              1130-1139 AREA1133     00189 1634     none     3
  CE CAPE                      1140-1149 AREA1143     00189 1652     none     3
  CF Ozone                     1150-1159 AREA1151     00189 1652     none     3
  CI GOES-E/W IR Composite       80-89       none        none        none     3
  CV GOES-E/W VIS Composite      90-99       none        none        none     3
  CW GOES-E/W H2O Composite      70-79       none        none        none     3
  LD NLDN Lightning Flashes      71-71       none        none        none     3
  MA Surface MD data            default      none        none    SFC.BAT      3
  N1 GOES-East IR/TOPO Composi  220-229      none        none        none     3
  N2 GOES-East VIS/TOPO Compos  230-239      none        none        none     3
  N3 GOES-West IR/TOPO Composi  240-249      none        none        none     3
  N4 GOES-West VIS/TOPO Compos  250-259      none        none        none     3
  N5 MDR/TOPO Composite         260-269      none        none        none     3
  N6 Mollweide IR/TOPO Composi  270-279      none        none        none     3
  N7 GOES-E/W IR/TOPO Composit  280-289      none        none        none     3
  N8 GOES-E/W VIS/TOPO Composi  290-299      none        none        none     3
  NF Global Initialization Gri  101-102      none        none    GLOBAL.BAT   3
  NG Early Domestic Products      1-2        none        none    ADDGRID.BAT  3
  R1 Base Reflectivity Tilt 1   300-339  AREA0300     00189 1736     none     3
  R2 Base Reflectivity Tilt 2   340-379  AREA0379     00189 1738     none     3
  R3 Base Reflectivity Tilt 3   380-419  AREA0419     00189 1740     none     3
  R4 Base Reflectivity Tilt 4   420-459  AREA0459     00189 1743     none     3
  R5 Composite Reflectivity     460-499  AREA0498     00189 1734     none     3
  R6 Layer Reflect SFC-24 K ft  500-539  AREA0538     00189 1734     none     3
  R7 Layer Reflect 24-33 K ft   540-579  AREA0555     00189 1336     none     3
  R8 Layer Reflect 33-60 K ft   580-619  AREA0594     00189 1330     none     3
  R9 Echo Tops                  620-659  AREA0635     00189 1336     none     3
  RA Vertical Liquid H2O        660-699  AREA0674     00189 1330     none     3
  RB 1-hour Surface Rain Total  700-739      none        none        none     3
  RC 3-hour Surface Rain Total  740-779  AREA0741        none        none     3
  RD Storm Total Rainfall       780-819      none        none        none     3
  RE Radial Velocity Tilt 1     820-859  AREA0858     00189 1737     none     3
  RF Radial Velocity Tilt 2     860-899  AREA0861     00189 1739     none     3
  RG Radial Velocity Tilt 3     900-939  AREA0939     00189 1742     none     3
  RH Radial Velocity Tilt 4     940-979  AREA0979     00189 1743     none     3
  RI 248 nm Base Reflectivity   980-1019 AREA1019     00189 1736     none     3
  RJ Storm-Rel Mean Vel Tilt 1 1020-1059 AREA1034     00189 1332     none     3
  RK Storm-Rel Mean Vel Tilt 2 1060-1099 AREA1076     00189 1332     none     3
  RM Mandatory Upper Air MD da  default      none        none    MAN.BAT      3
  RS Significant Upper Air MD   default      none        none    SIG.BAT      3
  U1 Antarctic Composite        190-195  AREA0190     00189 1511     none     3
  U2 FSL2 hourly wind profiler  default  MDXX0089     00189 1718     none     3
  U3 Manually Digitized Radar   200-205  AREA0204     00189 1706     none     3
s U4 Unidata-Wisconsin hourly   default      none        none    PROFILER.BAT 3
  U5 GOES-9 North America IR    130-135  AREA0134     00189 1717     none     3
  U6 FSL2 6-minute Wind profil  default  MDXX0099     00189 1741     none     3
  U9 GOES-9 North America VIS   120-125  AREA0120     00189 1718     none     3
  UA Educational Floater        160-165  AREA0164     00189 1732     none     3
  UB GOES-9 Water Vapor         170-175  AREA0174     00189 1716     none     3
  UC Educational Floater II      60-69   AREA0062     00189 1734     none     3
  UI GOES-8 North America IR    150-159  AREA0152     00189 1736     none     3
  UM Administrative Message     default      none        none        none     1
  UR Research Floater           180-185      none        none        none     3
  US Undecoded SAO Data         default      none        none        none     1
  UV GOES-8 North America VIS   140-149  AREA0143     00189 1737     none     3
  UW GOES-8 North America H2O   210-219  AREA0212     00189 1736     none     3
  UX Mollweide Composite IR     100-109  AREA0107     00189 1630 MOLL.BAT     3
  UY Mollweide Composite H2O    110-119  AREA0117     00189 1636     none     3
route.k: Done

Here is at least one problem.  The routing table does not have PostProcess
BATCH files defined for any image product other than UX, the Mollweide
Composite IR.  The question is how this happened?

OK, I see the problem.  You created a new routing table definiton BATCH
file, ~mcidas/data/LROUTE.BAT whose last engries for image products
have no PostProcess BATCH files defined.  Here is the end of this file:

REM New Compression
ROUTE ADD CA AREA 1100 1109 CC=3 SYS=2119 2120 2121 "Cloud Top Pressure
ROUTE ADD CB AREA 1110 1119 CC=3 SYS=2110 2111 2112 "Precipitable Water
ROUTE ADD CC AREA 1120 1129 CC=3 SYS=2113 2114 2115 "Sea Surface Temperature
ROUTE ADD CD AREA 1130 1139 CC=3 SYS=2116 2117 2118 "Lifted Index
ROUTE ADD CE AREA 1140 1149 CC=3 SYS=2101 2102 2103 "CAPE
ROUTE ADD CF AREA 1150 1159 CC=3 SYS=2104 2105 2106 "Ozone
ROUTE REL CA CB CC CD CE CF
ROUTE ADD U1 AREA 190 195 CC=3 SYS=2119 2120 2121 "Antarctic Composite
ROUTE ADD U3 AREA 200 205 CC=3 SYS=2110 2111 2112 "Manually Digitized Radar
ROUTE ADD U5 AREA 130 135 CC=3 SYS=2113 2114 2115 "GOES-9 North America IR
ROUTE ADD U9 AREA 120 125 CC=3 SYS=2116 2117 2118 "GOES-9 North America VIS
ROUTE ADD UA AREA 160 165 CC=3 SYS=2101 2102 2103 "Educational Floater
ROUTE ADD UB AREA 170 175 CC=3 SYS=2104 2105 2106 "GOES-9 Water Vapor
ROUTE ADD UI AREA 150 159 CC=3 SYS=2143 2144 2145 "GOES-8 North America IR
ROUTE ADD UR AREA 180 185 CC=3 SYS=2107 2108 2109 "Research Floater
ROUTE ADD UV AREA 140 149 CC=3 SYS=2146 2147 2148 "GOES-8 North America VIS
ROUTE ADD UW AREA 210 219 CC=3 SYS=2149 2150 2151 "GOES-8 North America H2O

ROUTE REL U1 U3 U5 U9 UA UB UI UR UV UW

I have the feeling that you really meant to _NOT_ include the ROUTE ADDs
for products U1, U3, U5, U9, UA, UB, UI, UR, UV, and UW as these are
already defined before the section for the CIMSS products.  I think that
if you either delete or comment out the entries for U1, etc at the end
of the file:

<login as mcidas>

cd data
<edit LROUTE.BAT and change the near end to:>
REM New Compression
ROUTE ADD CA AREA 1100 1109 CC=3 SYS=2119 2120 2121 "Cloud Top Pressure
ROUTE ADD CB AREA 1110 1119 CC=3 SYS=2110 2111 2112 "Precipitable Water
ROUTE ADD CC AREA 1120 1129 CC=3 SYS=2113 2114 2115 "Sea Surface Temperature
ROUTE ADD CD AREA 1130 1139 CC=3 SYS=2116 2117 2118 "Lifted Index
ROUTE ADD CE AREA 1140 1149 CC=3 SYS=2101 2102 2103 "CAPE
ROUTE ADD CF AREA 1150 1159 CC=3 SYS=2104 2105 2106 "Ozone
ROUTE REL CA CB CC CD CE CF
REM ROUTE ADD U1 AREA 190 195 CC=3 SYS=2119 2120 2121 "Antarctic Composite
REM ROUTE ADD U3 AREA 200 205 CC=3 SYS=2110 2111 2112 "Manually Digitized Radar
REM ROUTE ADD U5 AREA 130 135 CC=3 SYS=2113 2114 2115 "GOES-9 North America IR
REM ROUTE ADD U9 AREA 120 125 CC=3 SYS=2116 2117 2118 "GOES-9 North America VIS
REM ROUTE ADD UA AREA 160 165 CC=3 SYS=2101 2102 2103 "Educational Floater
REM ROUTE ADD UB AREA 170 175 CC=3 SYS=2104 2105 2106 "GOES-9 Water Vapor
REM ROUTE ADD UI AREA 150 159 CC=3 SYS=2143 2144 2145 "GOES-8 North America IR
REM ROUTE ADD UR AREA 180 185 CC=3 SYS=2107 2108 2109 "Research Floater
REM ROUTE ADD UV AREA 140 149 CC=3 SYS=2146 2147 2148 "GOES-8 North America VIS
REM ROUTE ADD UW AREA 210 219 CC=3 SYS=2149 2150 2151 "GOES-8 North America H2O

ROUTE REL U1 U3 U5 U9 UA UB UI UR UV UW
 ...

and then redo your routing table:

batch.k LROUTE.BAT

Then your routing table will have PostProcess defined for all of the various
image products.

Let's try that quickly and see what happens.  I am heading out for lunch
right now (actually, I play hoops at noon :-).  I will return at 1:15
my time and take a look.

After I return, I will get to your other comments below, and provide
some recommendations on redoing your setup.

Tom

>More in context below:
>
>At 04:30 PM 7/6/2000 -0600, you wrote:
>>>From: "James D. Marco" <address@hidden>
>>>Organization: Cornell
>>>Keywords: 200007062021.e66KLJT26042 McIDAS-X ROUTE PP BATCH
>>
>>Jdm,
>>
>>>     I seem to have a bit of a problem with the TOPO/COMPOSITE
>>>files, Fronts and Weather Watches.  
>>
>>Hmm... this could be two different problems, not one.
>Sounds right.  
>
>>>     I've gone through the McIDAS, LDM-McIDAS and LDM configurations
>>>a three times since I updated to McIDAS installation...no luck.
>Four times now...
>
>>
>>From this comment, I am assuming that things stopped working after
>>you did downloaded the full McIDAS-X version and compiled it.
>>the TOPO compositing failure, however, would more likely be
>>related to the ldm-mcidas update.
>Exactly correct.  I tweaked and played around for a couple of weeks before
>bothering you.
>
>>
>>>     The LDM and McIDAS were getting this OK, before the update.
>>
>>The topo compositing problem might be that your setup uses the
>>ldm-mcidas decoders "in place".  What I mean by this is that you don't
>>copy the decoders to a different directory, link ~ldm/decoders (or
>>~ldm/util).  If you do use them in place, then the Bourne shell script
>>file, batch.k, that is distributed with ldm-mcidas, is now the one
>>being used, and it is most likely not configured to run in your
>>environment.
>Yup.  They are copied directly into ~ldm/decoders.  I knew it had to be
>configured from your web stuff and it looks OK....'course it ALL looks OK
>to me.
>
>>
>>If this is not the case, we have to look at the viability of the
>>routines that are involved in the compositing process:  AA, MA, REMAP,
>>VIRTIMG.
>I assume these are source code tweaks?
>
>>>     Kind'o stymied. In the case of the TopoComposites, it appears that 
>>>they are not being built.
>The ROUTEPP.LOG shows this, as above.
>
>>What do the entries in your routing log file look like?  This would be
>>the file defined by MCLOG in the Bourne shell script batch.k.  The copy
>>of batch.k that is being used for the PostProcessing is the one that
>>will be found by a 'which batch.k' when logged in as the user running
>>your LDM.
>See above...
>
>>>In the case of the ASUSxxx files, they are 
>>>available(LDM is working), but nothing I do seems to get McIDAS to look at
>>>them.
>Ahhh, new info.  I was mistakenly under the impression that the XCD was 
>using these to build or index into the Front/Watch plots. Do I still need 
>these to be filed from ldm/pqact?
> 
>>So, you are using FRONT to display the fronts?  What routines are being
>>run that are failing.  The reason I ask is that we have been moving away
>>from the user of FRONT (which is not an ADDE routine) and moving to
>>use of FRNTDISP (which _is_ and ADDE routine.  FRNTDISP does not read
>>the ASUS/FSUS files at all.  It does, however, need to access the
>>RTWXTEXT dataset.  This is the same dataset that is needed for access
>>of the weather watches, so I am suspicious about the decoding of these.
>No, I don't use FRONT.  I use the FRNTDISP command in batch files. The 
>commands in the scripts are taken directly from the McIDASX/MCGUI text 
>window and modified. Sooo, Mcidas is using FRNTDISP, also. (This has been 
>a little flakey in the past, some data -for McIDAS frames- comes up
>missing.)  
>
>>What is the name of the machine running your ADDE server, and are you
>>running the ADDE remote server (so I can take a look from here)?
>vorticity.cit.cornell.edu (132.236.186.25)  I did an install for the
>McADDE stuff, but have never gotten around to configuring anything...
>or if I need too, soo I don't know if you can see this remotely, around 
>the TCP wrappers? See above... I still use nfs to distribute to my McIDAS
>servers.  A partitioned subnet localizes the net traffic to the Map room 
>and MAD lab (Meteorological and Atmospheric Data Lab), soo, this is not
>as bad as it sounds. The nets are all 100mbit on that partition.
>
>>>                                     Any thoughts?
>>
>>I am suspicious that the XCD decoding of the data for RTWXTEXT is not
>>working correctly.  This would account for fronts not being drawn
>>by FRNTDISP and no weather watch boxes.  The failure of the topo
>>compositing, however, would most likely be related to ldm-mcidas things.
>>Let's take these problems on individually.
>Good thought. The RTWXTEXT could be a problem.  The ingetext and
>run_xcd didn't want to come up after the rebuild.  (Couple of iterations
>of tweaks to the various pathing to get it running.)  
>
>>>                                             Thanks!
>>
>>Tom
>>***************************************************************************
>>Unidata User Support                                    UCAR Unidata
>>(303)497-8644                                                  P.O. Box
>>address@hidden                                   Boulder, CO
>>---------------------------------------------------------------------------
>>Unidata WWW Service                        http://www.unidata.ucar.edu/
>>***************************************************************************
>>
>