[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #HCO-891524]: Unidata Performance Question



Michael,

No, I've never heard of pqinsert(1) looping on a single file -- and it's used 
extensively by NCEP for the CONDUIT feed. According to the code, if a product 
can't be inserted, then pqinsert(1) logs the fact as an "internal error" and 
moves on to the next file.

What version of the LDM are you running?

Can you send me any relevant log entries?

> Steve,
> 
> I was wondering if you have ever had reports about a problem with pqinsert
> hanging and processing the same file over and over again.
> 
> We use a perl script here at UNAVCO to transfer files from our FTP-IN
> directory on our customer accessable server into LDM by calling pqinsert
> on the file. We let all the files age 10 minutes to ensure they are
> complete before calling the pqinsert. Yesterday, pqinsert got stuck
> on a file and locked up the entire LDM system. The log file shows that
> pqinsert added the same file to the queue over 80000 lines of logging.
> 
> I'm not even sure what might cause this kind of issue other than an
> incomplete file, or a file with some type of CRC error that would have
> changed the MD5 checksum every time.
> 
> At any rate I was wondering if you had heard of this before, and if you
> might have any mitigations you might suggest to keep it from happening.
> 
> Thanks
> -Mike


Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: HCO-891524
Department: Support LDM
Priority: Normal
Status: Closed