[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: data flow (fwd)

Jeff Weber wrote:
> 
> Hi again,
> 
> These seem to be pretty LDM specific, can you field these?
> 
> I hope so....
> 
> -Jeff
> 
> ---------- Forwarded message ----------
> Date: Thu, 11 Jan 2001 11:14:08 -0500
> From: James R. Frysinger <address@hidden>
> To: address@hidden
> Cc: Jeff Weber <address@hidden>
> Subject: data flow
> 
> On Tuesday I used notifyme to check the primary and secondary
> feedpoints assigned to us (fsu.edu and utah.edu, respectively) and was
> rewarde in each case by a seemingly no-ending stream of records,
> indicating various data types that are available. The notifyme command
> seems to result in a continuously scrolling output. Finally, I had to
> stop the executions manually (^z then kill). Is this normal response?

Yes, notifyme was coded to be stopped with <cntl-c>.  Many of the ldm
programs are terminated this way.

> 
> I concluded that we are in the 'allow' sections at each site. When I
> launched LDM yesterday (Wednesday) my ldm.pq file rapidly filled to
> slightly over 100 Mb. It now stays at that size and the time stamp
> updates every minute. That makes me think that we are getting some sort
> of data. However, that's the only indication I have that data is coming
> in. Statistics are not building. Executing ldmadmin log results in null
> output. I created subdirectories in my data directory to match the
> decoder output directories in my MCIDAS decoder (pnga2area) listings in
> pqact. That stopped the error messages that scour was generating but
> resulted in no data files. Should I also create a subdirectroy with the
> structure data/2/4/6/ to collect WMO data?

You can use 'ldmadmin watch' to see each product that comes into your
queue.  'ldmadmin watch' is simply a call to the pqutil program.  If you
only want to see products from a particular feed type, you can do
'pqutil -w -f MCIDAS ~ldm/data/ldm.pq', for example.

(Or, you could do a notifyme to your own machine.  Again, if you want to
see just products from a particular feed type you can do 'notifyme -v -l
- -h <yourHost> -f <feedType>')

Your queue size should remain fixed from the time that the queue was
created.  A 100Mb queue is relatively small.  Typically users want their
queue to hold one hour's worth of data.  With the WMO feed, one hour's
worth is more likely to be around 300 - 500Mb.  You'll have to play
around to determine the proper queue size for your site.  Let the LDM
run for a while, then use 'pqmon' to determine how the space in your
queue is being used.

It sounds like your logging isn't working.  That's an important piece to
get working, so you can diagnose other problems.  Does the log file
exist already?  Did you following the instructions in
http://www.unidata.ucar.edu/packages/ldm/ldmPreInstallList.html under
"Configuring the Operating System as Root"?  The other piece is that the
log file must exist, even if it's a zero length file.  Does it?

It sounds like your pqact is having problems if it's not writing data. 
Once you get logging working, programs like pqact can report problems. 
Or, you can invoke pqact verbosely from the command line and have it log
to the display to see what it's doing: "pqact -v -l - ".

You do not need to create subdirectories for pqact - it will create the
directories it needs as it needs them.

Statistics will be gathered during the hour.  Are you .stats files being
updated at all?

> 
> At the present, my pqact file conains only  the WMO, MCIDAS, and NLDN
> data types. My ldmd.conf file requests only the UNIDATA feed from my
> feedpoints. Lines are in there for NLDN but are commented out at the
> moment.
> 
> I plan to stop LDM, uncomment the NLDN lines, and restart LDM. I also
> plan to add GEMPAK decoders to my pqact file, though I'm not sure if
> LDM needs to be stopped for that.
> 
> Can you provide me with any guidance at this point? Does my plan sound
> reasonable? Should I also include any other data types in my ldmd.conf,
> such as GEMPAK, or such like?
> 
> Jim Frysinger
> 

The approach that has worked best for me is to get small things working,
then add on.  First, make sure logging is working.  Whenever you start
the ldm you should see at least one line in the log that says something
like this line from my own ldmd.log:
Jan 08 22:23:37 imogene rpc.ldmd[32623]: Starting Up (built: Sep  8 2000
10:54:22) 

which 'ldmadmin log' should show you.

Then make sure you're getting data into your queue with 'ldmadmin
watch'.  

Then start first with just the WMO line to pqact.conf.  Start with
simply trying to file data.

Once you get that working successfully then try adding other feed types
and piping data into decoders.

Hope this helps!

By the way, if you have more questions like these, it's best to send
them to address@hidden. Jeff handles getting people set up.  I
handle more ldm-specific issues.  Mail to support is forwarded to the
appropriate person depending on who's available.

Anne
> --
> James R. Frysinger                  University/College of Charleston
> 10 Captiva Row                      Dept. of Physics and Astronomy
> Charleston, SC 29407                66 George Street
> 843.225.0805                        Charleston, SC 29424
> http://www.cofc.edu/~frysingj       address@hidden
> Cert. Adv. Metrication Specialist   843.953.7644

-- 
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                 P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************