[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #SBX-276608]: LDM crashed again



Gilbert,

Did process 5045 dump core? If so, would you please send me a stack trace.

> Hi Steve,
> 
> LDM 6.12.7 betaa (early version) crashed on
> my NOAAport ingest server. I've been so busy,
> I didn't even know it had been down for a while...
> and nobody noticed since they autofailed over to my other servers.
> 
> ----------------------------------------------------------------------
> 
> Dec 10 10:06:37 noaaport ldm-relay1.tamu.edu(feed)[5004] NOTE: Starting
> Up(6.12.7/6): 20141210090636.162 TS_ENDT {{NGRID,  ".*"}},
> SIG=41b89ec8a803c9a1158d5df941065f76, Alternate
> Dec 10 10:06:37 noaaport ldm-relay1.tamu.edu(feed)[5004] NOTE: topo:
> ldm-relay1.tamu.edu {{NGRID, (.*)}}
> Dec 10 10:06:37 noaaport ldm01.allisonhouse.com(feed)[5006] NOTE:
> Data-product with signature da1c6eb37c5b5e7008fecadae99eb7e9 wasn't found
> in product-queue
> Dec 10 10:06:37 noaaport ldm01.allisonhouse.com(feed)[5006] NOTE: Starting
> Up(6.12.7/6): 20141210094136.623 TS_ENDT {{IDS|DDPLUS,  ".*"}},
> SIG=da1c6eb37c5b5e7008fecadae99eb7e9, Alternate
> Dec 10 10:06:37 noaaport ldm01.allisonhouse.com(feed)[5006] NOTE: topo:
> ldm01.allisonhouse.com {{IDS|DDPLUS, (.*)}}
> Dec 10 10:07:17 noaaport ec2-107-21-238-6.compute-1.amazo[5045] NOTE:
> [uldb.c:1523] Terminated obsolete upstream LDM (addr=107.21.238.6,
> pid=4977, vers=6, type=feeder, mode=primary, sub=(20141210090939.545
> TS_ENDT {{NEXRAD3,  "/p(NCR)"}}))
> Dec 10 10:07:17 noaaport ec2-107-21-238-6.compute-1.amazo[4977] NOTE:
> Exiting
> Dec 10 10:07:18 noaaport ldmd[4843] NOTE: child 5045 terminated by signal
> 11
> Dec 10 10:07:18 noaaport ldmd[4843] NOTE: Killing (SIGTERM) process group
> Dec 10 10:07:18 noaaport ldmd[4843] NOTE: Exiting
> Dec 10 10:07:18 noaaport sasquatch.tamu.edu(feed)[4935] NOTE: Exiting
> Dec 10 10:07:18 noaaport ldm-relay1.tamu.edu(feed)[5004] NOTE: Exiting
> Dec 10 10:07:18 noaaport ldm-relay0.tamu.edu(feed)[5003] NOTE: Exiting
> Dec 10 10:07:18 noaaport ldm01.allisonhouse.com(feed)[5000] NOTE: Exiting
> Dec 10 10:07:18 noaaport ldm01.allisonhouse.com(feed)[5006] NOTE: Failure;
> COMINGSOON: RPC: Unable to receive; errno = Bad file descriptor
> Dec 10 10:07:18 noaaport www2.ehs.niu.edu(feed)[4924] NOTE: Exiting
> Dec 10 10:07:18 noaaport ldm01.allisonhouse.com(feed)[5006] NOTE: Exiting
> Dec 10 10:07:18 noaaport ldm-relay0.tamu.edu(feed)[5002] NOTE: Failure;
> COMINGSOON: RPC: Unable to receive; errno = Bad file descriptor
> Dec 10 10:07:18 noaaport noaaportIngester[4853] NOTE:
> ----------------------------------------
> Dec 10 10:07:18 noaaport noaaportIngester[4852] NOTE:
> ----------------------------------------
> Dec 10 10:07:18 noaaport ldm-relay0.tamu.edu(feed)[5002] NOTE: Exiting
> Dec 10 10:07:18 noaaport ldm01.allisonhouse.com(feed)[5001] NOTE: Exiting
> Dec 10 10:07:18 noaaport rtstats[4855] NOTE: Exiting
> Dec 10 10:07:18 noaaport idd.unidata.ucar.edu[4860] NOTE: Exiting
> Dec 10 10:07:18 noaaport noaaportIngester[4851] NOTE:
> ----------------------------------------
> Dec 10 10:07:18 noaaport weather2.admin.niu.edu[4859] NOTE: Exiting
> Dec 10 10:07:18 noaaport idd.unidata.ucar.edu[4858] NOTE: Exiting
> Dec 10 10:07:18 noaaport idd.unidata.ucar.edu[4857] NOTE: Exiting
> Dec 10 10:07:18 noaaport idd.unidata.ucar.edu[4856] NOTE: Exiting
> Dec 10 10:07:18 noaaport ldmd[4843] NOTE: Terminating process group
> Dec 10 10:07:18 noaaport ldm-relay1.tamu.edu(feed)[5005] NOTE: Failure;
> COMINGSOON: RPC: Unable to receive; errno = Bad file descriptor
> Dec 10 10:07:18 noaaport noaaportIngester[4848] NOTE:
> ----------------------------------------
> Dec 10 10:07:18 noaaport ec2-107-21-238-6.compute-1.amazo[4978] NOTE:
> Failure; COMINGSOON: RPC: Unable to receive; errno = Bad file descriptor
> Dec 10 10:07:18 noaaport sasquatch.tamu.edu(feed)[4957] NOTE: Failure;
> COMINGSOON: RPC: Unable to receive; errno = Bad file descriptor
> Dec 10 10:07:18 noaaport noaaportIngester[4846] NOTE:
> ----------------------------------------
> Dec 10 10:07:18 noaaport noaaportIngester[4847] NOTE:
> ----------------------------------------
> Dec 10 10:07:18 noaaport noaaportIngester[4849] NOTE:
> ----------------------------------------
> Dec 10 10:07:18 noaaport noaaportIngester[4850] NOTE:
> ----------------------------------------
> Dec 10 10:07:18 noaaport noaaportIngester[4853] NOTE: Ingestion
> Statistics:
> Dec 10 10:07:18 noaaport noaaportIngester[4852] NOTE: Ingestion
> Statistics:
> Dec 10 10:07:18 noaaport noaaportIngester[4851] NOTE: Ingestion
> Statistics:
> Dec 10 10:07:18 noaaport ldmd[4843] NOTE: child 5002 exited with status 6
> Dec 10 10:07:18 noaaport ldm-relay1.tamu.edu(feed)[5005] NOTE: Exiting
> Dec 10 10:07:18 noaaport noaaportIngester[4848] NOTE: Ingestion
> Statistics:
> Dec 10 10:07:18 noaaport ec2-107-21-238-6.compute-1.amazo[4978] NOTE:
> Exiting
> Dec 10 10:07:18 noaaport sasquatch.tamu.edu(feed)[4957] NOTE: Exiting
> Dec 10 10:07:18 noaaport noaaportIngester[4846] NOTE: Ingestion
> Statistics:
> Dec 10 10:07:18 noaaport noaaportIngester[4847] NOTE: Ingestion
> Statistics:
> Dec 10 10:07:18 noaaport noaaportIngester[4849] NOTE: Ingestion
> Statistics:
> Dec 10 10:07:18 noaaport noaaportIngester[4850] NOTE: Ingestion
> Statistics:
> Dec 10 10:07:18 noaaport noaaportIngester[4853] NOTE:     Since Previous
> Report (or Start):
> Dec 10 10:07:18 noaaport noaaportIngester[4852] NOTE:     Since Previous
> Report (or Start):
> Dec 10 10:07:18 noaaport noaaportIngester[4851] NOTE:     Since Previous
> Report (or Start):
> Dec 10 10:07:18 noaaport ldmd[4843] NOTE: child 5006 exited with status 6
> Dec 10 10:07:18 noaaport noaaportIngester[4848] NOTE:     Since Previous
> Report (or Start):
> Dec 10 10:07:18 noaaport noaaportIngester[4846] NOTE:     Since Previous
> Report (or Start):
> Dec 10 10:07:18 noaaport noaaportIngester[4847] NOTE:     Since Previous
> Report (or Start):
> 
> Gilbert
> 
> *******************************************************************************
> Gilbert Sebenste                                                    ********
> (My opinions only!)                                                  ******
> Staff Meteorologist, Northern Illinois University                      ****
> E-mail: address@hidden                                  ***
> web: http://weather.admin.niu.edu                                      **
> Twitter: http://www.twitter.com/NIU_Weather                            **
> Facebook: http://www.facebook.com/niu.weather                           *
> *******************************************************************************

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: SBX-276608
Department: Support LDM
Priority: Normal
Status: Closed