[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20000803: ldm hang

Mike,

The assertion failed message is generally a sign of a corrupt product
queue. If the queue is corrupt, you should delete the queue and remake it
before restarting the LDM, otherwise, you will dump out of LDM when
it hits that same region.

Is the problem repeating after you have created a new queue?

Steve Chiswell
Unidata User Support



>From: Mike Trexler <address@hidden>
>Organization: UCAR/Unidata
>Keywords: 200008031839.e73IdhT14536

>We've had a problem with the ldm hanging up for the past
>couple of days at approximately the same time (~22UTC).
>I've attached the log file to see if you can help me 
>understand what might be going wrong.
>
>thanks for your help,
>Mike
>
>Aug 02 21:45:04 meaculpa.nrrc.ncsu.edu sysu1[21014]: Exiting
>Aug 02 21:45:04 meaculpa.nrrc.ncsu.edu sysu1[21016]: Connection reset by peer
>Aug 02 21:45:04 meaculpa.nrrc.ncsu.edu sysu1[21016]: Exiting
>Aug 02 21:45:10 meaculpa.nrrc.ncsu.edu sysu1[29904]: Connection from sysu1.uni
> .wsicorp.com
>Aug 02 21:45:10 meaculpa.nrrc.ncsu.edu sysu1[29905]: Connection from sysu1.uni
> .wsicorp.com
>Aug 02 21:45:10 meaculpa.nrrc.ncsu.edu sysu1[29904]: hiya: 20000802221753.975 
> TS_ENDT {{WSI,  ".*"}}
>Aug 02 21:45:10 meaculpa.nrrc.ncsu.edu sysu1[29905]: hiya: 20000802221753.975 
> TS_ENDT {{WSI,  ".*"}}
>Aug 02 21:45:15 meaculpa.nrrc.ncsu.edu lwtoa3[66]: PRODUCT CODE=U9          21
> 5         221617
>Aug 02 21:45:57 meaculpa.nrrc.ncsu.edu lwtoa3[66]:  Done -- AREA= 128
>Aug 02 21:46:07 meaculpa.nrrc.ncsu.edu pqact[15284]: pbuf_flush (18) write: Br
> oken pipe
>Aug 02 21:46:07 meaculpa.nrrc.ncsu.edu pqact[15284]: pipe_dbufput: /meaculpa/n
> awips/bin/sol/dcncprof-v-l/newdata/gempak/Logs/dcncp
>rof.log-eGEMTBL=/me
>Aug 02 21:46:07 meaculpa.nrrc.ncsu.edu pqact[15284]: pipe_prodput: trying agai
> n
>Aug 02 21:46:08 meaculpa.nrrc.ncsu.edu pqact[15284]: pbuf_flush (18) write: Br
> oken pipe
>Aug 02 21:46:08 meaculpa.nrrc.ncsu.edu pqact[15284]: pipe_dbufput: /meaculpa/n
> awips/bin/sol/dcncprof-v-l/newdata/gempak/Logs/dcncp
>rof.log-eGEMTBL=/me
>Aug 02 21:46:08 meaculpa.nrrc.ncsu.edu pqact[15284]: child 1170 exited with st
> atus 127
>Aug 02 21:46:08 meaculpa.nrrc.ncsu.edu pqact[15284]: child 1168 exited with st
> atus 127
>Aug 02 21:46:33 meaculpa.nrrc.ncsu.edu pqexpire[15282]: > Recycled  46001.151 
> kb/hr (  4888.012 prods per hour)
>Aug 02 21:50:03 meaculpa.nrrc.ncsu.edu sysu1[29905]: Connection reset by peer
>Aug 02 21:50:03 meaculpa.nrrc.ncsu.edu sysu1[29905]: Exiting
>Aug 02 21:50:03 meaculpa.nrrc.ncsu.edu sysu1[29904]: Connection reset by peer
>Aug 02 21:50:03 meaculpa.nrrc.ncsu.edu sysu1[29904]: Exiting
>Aug 02 21:50:09 meaculpa.nrrc.ncsu.edu sysu1[8262]: Connection from sysu1.uni.
> wsicorp.com
>Aug 02 21:50:09 meaculpa.nrrc.ncsu.edu sysu1[8264]: Connection from sysu1.uni.
> wsicorp.com
>Aug 02 21:50:09 meaculpa.nrrc.ncsu.edu sysu1[8262]: hiya: 20000802222258.287 T
> S_ENDT {{WSI,  ".*"}}
>Aug 02 21:50:09 meaculpa.nrrc.ncsu.edu sysu1[8264]: hiya: 20000802222258.287 T
> S_ENDT {{WSI,  ".*"}}
>Aug 02 21:51:03 meaculpa.nrrc.ncsu.edu pqact[15284]: pbuf_flush (9) write: Bro
> ken pipe
>Aug 02 21:51:03 meaculpa.nrrc.ncsu.edu pqact[15284]: pipe_dbufput: /meaculpa/n
> awips/bin/sol/dcncprof-v-l/newdata/gempak/Logs/dcncp
>rof.log-eGEMTBL=/me
>Aug 02 21:51:03 meaculpa.nrrc.ncsu.edu pqact[15284]: pipe_prodput: trying agai
> n
>Aug 02 21:51:03 meaculpa.nrrc.ncsu.edu pqact[15284]: pbuf_flush (9) write: Bro
> ken pipe
>Aug 02 21:51:03 meaculpa.nrrc.ncsu.edu pqact[15284]: pipe_dbufput: /meaculpa/n
> awips/bin/sol/dcncprof-v-l/newdata/gempak/Logs/dcncp
>rof.log-eGEMTBL=/me
>Aug 02 21:51:03 meaculpa.nrrc.ncsu.edu pqact[15284]: child 9671 exited with st
> atus 127
>Aug 02 21:51:03 meaculpa.nrrc.ncsu.edu pqact[15284]: child 9666 exited with st
> atus 127
>Aug 02 21:52:15 meaculpa.nrrc.ncsu.edu pqexpire[15282]: > Recycled  46019.303 
> kb/hr (  4978.871 prods per hour)
>Aug 02 21:55:04 meaculpa.nrrc.ncsu.edu sysu1[8262]: Connection reset by peer
>Aug 02 21:55:04 meaculpa.nrrc.ncsu.edu sysu1[8262]: Exiting
>Aug 02 21:55:04 meaculpa.nrrc.ncsu.edu sysu1[8264]: Connection reset by peer
>Aug 02 21:55:04 meaculpa.nrrc.ncsu.edu sysu1[8264]: Exiting
>Aug 02 21:55:10 meaculpa.nrrc.ncsu.edu sysu1[16735]: Connection from sysu1.uni
> .wsicorp.com
>Aug 02 21:55:10 meaculpa.nrrc.ncsu.edu sysu1[16736]: Connection from sysu1.uni
> .wsicorp.com
>Aug 02 21:55:10 meaculpa.nrrc.ncsu.edu sysu1[16735]: hiya: 20000802222801.340 
> TS_ENDT {{WSI,  ".*"}}
>Aug 02 21:55:10 meaculpa.nrrc.ncsu.edu sysu1[16736]: hiya: 20000802222801.340 
> TS_ENDT {{WSI,  ".*"}}
>Aug 02 21:57:37 meaculpa.nrrc.ncsu.edu pqact[15284]: pbuf_flush (7) write: Bro
> ken pipe
>Aug 02 21:57:37 meaculpa.nrrc.ncsu.edu pqact[15284]: pipe_dbufput: /meaculpa/n
> awips/bin/sol/dcncprof-v-l/newdata/gempak/Logs/dcncp
>rof.log-eGEMTBL=/me
>Aug 02 21:57:37 meaculpa.nrrc.ncsu.edu pqact[15284]: pipe_prodput: trying agai
> n
>Aug 02 21:57:37 meaculpa.nrrc.ncsu.edu pqact[15284]: pbuf_flush (7) write: Bro
> ken pipe
>Aug 02 21:57:37 meaculpa.nrrc.ncsu.edu pqact[15284]: pipe_dbufput: /meaculpa/n
> awips/bin/sol/dcncprof-v-l/newdata/gempak/Logs/dcncp
>rof.log-eGEMTBL=/me
>Aug 02 21:57:37 meaculpa.nrrc.ncsu.edu pqact[15284]: child 21502 exited with s
> tatus 127
>Aug 02 21:57:37 meaculpa.nrrc.ncsu.edu pqact[15284]: child 21499 exited with s
> tatus 127
>Aug 02 21:57:45 meaculpa.nrrc.ncsu.edu pqexpire[15282]: assertion "info->sz <=
>  xdrs->x_handy" failed: file "pq.c", line 4021
>Aug 02 21:57:50 meaculpa.nrrc.ncsu.edu lwtoa3[21842]: PRODUCT CODE=UX         
>  215         223035
>Aug 02 21:57:51 meaculpa.nrrc.ncsu.edu rpc.ldmd[15275]: child 15282 terminated
>  by signal 6
>Aug 02 21:57:51 meaculpa.nrrc.ncsu.edu rpc.ldmd[15275]: Killing (SIGINT) proce
> ss group
>Aug 02 21:57:51 meaculpa.nrrc.ncsu.edu rpc.ldmd[15275]: Interrupt
>Aug 02 21:57:51 meaculpa.nrrc.ncsu.edu rpc.ldmd[15275]: Exiting
>Aug 02 21:57:51 meaculpa.nrrc.ncsu.edu sysu1[16736]: Interrupt
>Aug 02 21:57:51 meaculpa.nrrc.ncsu.edu sysu1[16736]: Exiting
>Aug 02 21:57:51 meaculpa.nrrc.ncsu.edu lwtoa3[21842]: Interrupt
>Aug 02 21:57:51 meaculpa.nrrc.ncsu.edu sysu1[16735]: Interrupt
>Aug 02 21:57:51 meaculpa.nrrc.ncsu.edu sysu1[16735]: Exiting
>Aug 02 21:57:51 meaculpa.nrrc.ncsu.edu DCSYNOP[6731]: Interrupt Signal
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu pqact[15284]: Interrupt
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu pqbinstats[15283]: Interrupt
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu pqact[15284]: Exiting
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu pluto[15285]: Interrupt
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu pluto[15285]: Exiting
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu DCUAIR[6829]: Interrupt Signal
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu DCSYNOP[6788]: Interrupt Signal
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu DCHRLY[6724]: Interrupt Signal
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu udp.ldmd[15287]: Interrupt
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu striker[15286]: Interrupt
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu pqbinstats[15283]: Exiting
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu striker[15286]: Exiting
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu udp.ldmd[15287]: Exiting
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu rpc.ldmd[15275]: Terminating process gr
> oup
>Aug 02 21:57:52 meaculpa.nrrc.ncsu.edu DCSYNOP[6788]: Terminate Signal
>
>
>__________________________________________________
>**************************************************
>******          C. Michael Trexler          ******
>******       N C State Univ. - Box 8208     ****** 
>******  Dept. of Mar/Earth/Atmos. Sciences  ******
>******        Research III   Rm. 100        ******
>******        Raleigh, NC 27695-8208        ****** 
>******        (919) 515-1447  Phone         ******   
>******         (919) 515-1683  Fax          ******
>**************************************************
>--------------------------------------------------
>******        address@hidden         ******
>__________________________________________________
>