[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: pqinsert

Daryl Herzmann wrote:
> 
> Hello All,
>         I am trying to use ldm to send large gempak files between
> machines. I am currently using pqinsert.  The problem is that only a
> portion of the file gets transfered to the other machine. I believe that
> the queue is sufficently big on both machines (100 MB). THe only other
> products on the two machines are small text files.
>         For example, I just tried to insert a 10.4 MB file on one end and
> the other end a 2.1 MB file appears.  The pqinsert command did not
> report back any errors, so I assume that the queue could handle
> it. Anybody have any ideas, or am I approaching this wrong?
>         My goal is to create a distributed model processing system and I
> wanted to use ldm as the data collector and distributor, but this large
> file problem has got me puzzled. Thanks for your time.
> 
> Thanks,
>         Daryl
> 
>                                                     __   ,,,,
>                                                    /  \_|$$$$|__,
>                                                    |     ==========}
> ,,,,,, Daryl Herzmann  [ JR - MTEOR ]   ,,,,,,,,   +--+ JOHN    |_
> !!!!!! address@hidden              !!!!!!!! ___  |  DEERE ,_/
> $$$$$$ www.public.iastate.edu/~akrherz  $$$$$$$$/__|==-__-----_|
> |||||| ICQ: 25560887 AOL: "Daryl Herz"  ||||||||_______{}_____o_____

Hi Daryl,

I might be able to help you with your problem, but
I need some more information.

Take a look at the logs on both machines for any
relevant messages:  
  ldmadmin log | more
You could send them, or a portion of them, to me
if you wish.  Check the logs frequently as you try
the tests below.  They are a great window into
what's going on.

Here are a few other things to try:

- Before pqinserting the product, start a
'notifyme' to the remote machine using the
feedtype of the gempak file.  
   notifyme -vl - -f <yourFeedType> -h
<remoteMachineName>
As notifyme runs it will report all products
arriving under that feedtype.  So, if the product
arrives at the remote machine after pqinserting it
on the ingest machine you should see
notification.  (You can also use notifyme to look
back in time by using the -o option.)  If you get
the notification that the product arrived, then
perhaps the problem is in extracting the file from
the queue on the remote machine and filing it.

- On the ingest machine, after doing pqinsert, use
'pqcat' to see whether the product got into the
queue properly on that machine.  
  pqcat -vl - -f <yourFeedType> > /dev/null

For testing purposes, if you need to reinsert the
product multiple times to test the transmission to
your remote machine, you can use 'pqexpire -a 0 -f
<yourFeedType>' to remove it (and all other
products of that feedtype).  Otherwise, if it's
already in the queue you will get a message saying
it was a duplicate that was not reinserted into
the queue.

- Are you sure the connection between the two
machines is good?  Try a few 'traceroutes' just to
make sure.  If both machines are on a local
network, this is probably not the problem, but the
question is worth asking.

- Are you sure there's enough disk space on the
remote machine for the file?  I would assume the
permissions are set correctly if at least a
portion of the file made it over.

Try these things and let me know what you find.

Anne
-- 
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                 P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server      
http://www.unidata.ucar.edu/
****************************************************