[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[SCOOP #ZJL-127049]: rtstats -v isn't behaving as expected

Gerry, Donna et al,

Donna is correct.

The rtstats program was designed (as pqbinstats) to provide a snapshot
of the running LDM and not a log of all products received.

To log every EXP product received to check the monitor receipt and latency, 
you could use a pqact action such as:


EXP     (.*)
        EXEC    util/product_log prodlog/exp.%Y%m%d%H %Y%m%d%H%M%S "\1"

This expects a program/script called ~ldm/util/product_log to exist, and
3 items will be passed as input:

1) a directory/file name (I have an hourly log file above)
2) The date/time that pqact obtains from the LDM product itself. This
is not the current system time!
3) The complete ldm product name. Note the quotes in case spaces exist in the 
product name.


The simplest script ~ldm/util/product_log could be something like:
#!/bin/csh -f

#
# Log all products from LDM. Use pqact action:
# EXEC  util/product_log outfile ldmtime ldmname

if ( $#argv != 3 ) then
   echo "Incorrect number or arguments $#argv $argv" | \
        logger -t "$0 [$$]" -p local0.notice
endif

set LOGFIL="data/$1"

if ( ! -e $LOGFIL:h ) mkdir -p $LOGFIL:h
echo '['`date -u +"%Y%m%d%H%M%S"`']' '['${2}']' $3 >>! $LOGFIL
exit 0
#################################################################


This script will use the passed directory file name (and place it under the 
~ldm/data directory).
It will append a line for each product that contains:
[system time] [ldmproduct time passed as argument 2] "product name from 
argument 3"


Now, you could have a cron that would send these hourly files back to a central 
location using ldmsend
(which is basically what rtstats is doing).

In reality, if you had a very large number of files, you wouldn't want to use a 
script doing disk
IO as I have here, but it gets the point accross on what you would need to 
monitor latency of
the ldmproduct time vs the time pqact gets around to processing the EXEC line. 
A specialized version
of pqact (which is really what rtstats is too) could be made to dbfile this 
information, or make log
products to send to the central site.


Steve CHiswell
Unidata User Support






> Gerry, It appears that we can't obtain individual product receipt
> information as mentioned in the rtstats man page:
> > OPTIONS
> >        -v     Verbose logging.  A line is emitted for every product in
> > the queue whose feed type and identifier match the specified
> >               feedtype and pattern.  The emitted line contains the UTC
> > date and time, the program name, the  product  ingest  time,
> >               feedtype, sequence number, size in bytes, and the
> > product identifier.
> Looks like we will have to look elsewhere.
> 
> Donna


Ticket Details
===================
Ticket ID: ZJL-127049
Department: Support IDD SCOOP
Priority: Normal
Status: Closed