[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
Re: 20001201: pbuf_flush and pq_sequence failed log entries
- Subject: Re: 20001201: pbuf_flush and pq_sequence failed log entries
- Date: Mon, 04 Dec 2000 16:14:11 -0700
Unidata Support wrote:
>
> ------- Forwarded Message
>
> >To: address@hidden
> >From: Gregory Grosshans <address@hidden>
> >Subject: pbuf_flush and pq_sequence failed log entries
> >Organization: UCAR/Unidata
> >Keywords: 200012011829.eB1ITfo11872
>
> --------------0D92473C1124E093F8F0FA4C
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
>
> I'm obtaining some pbuf_flush log messages and was wondering if you can
> provide any input on what may be causing the log messages. Its been
> going on for at least seven days.
> The ldm queue is 750 MB and the system is ingesting the NOAAPORT NWSTG,
> GOES-EAST & WEST data streams as well as a UNISYS radar feed containing
> single site radar and mosaic data.
>
> The machine is a HP J5000 with 2 GB of RAM. The LDM queue is on a local
> SCSI disk drive. Any data stored or decoded and then stored is written
> to a NFS mount (a NetAPP NFS fileserver). The sysadmin checked and the
> local disks were not that busy at approximately 1620Z when the
> pbuf_flush entries were logged.
>
> A similar log message occurred last spring with correspondence with
> you. That was happening on a slower server with disks very busy. This
> pbuf log entry hasn't caused any
> negative impacts I'm aware of (corrupted gempak decoder files),
> truncated data, etc.
>
> Also, what is a pq_sequence failed log message indicate?
>
> > Dec 01 18:07:28 alice as2(feed)[13371]: pq_sequence failed: I/O error
> > (errno = 5)
> >
>
> Thanks,
> Gregg Grosshans
> SPC
>
Hi (again) Gregg!
From our support archives:
> >The pbuf_flush 15 means that the ACTION ie ( FILE, STDIO, PIPE, etc )
> >could not write to the destinations because the resource was not available
> >at that time or it was blocked. The log entry ususally occurs when disk
> >activity is at it's peak for some reason. It could be the LDM doing many
> >disk writes or another process tying up the disk.
(See
http://www.unidata.ucar.edu/cgi-bin/mfs/65/3580?11#mfs.)
Offhand, I can think of two reasons this might occur:
- you're piping to a decoder and the decoder is slow
- you're filing to a disk and the disk is slow. Or a
decoder is writing to a slow disk, thus making the decoder
slow (as above). I see your sys admin found the local disks
not too busy, but what about the NSF mounted disks that the
decoders are writing to?
Are you getting more data causing more decoding or more
filing? Can you think of anything that changed 7 days ago?
If these don't answer the problem, do try a search in our
support archives:
http://www.unidata.ucar.edu/glimpsedocs/ghldm.html. There
are lots of messages in there involving pbuf_flush. If that
doesn't help you, let me know.
The pq_sequence failure occurs when the LDM is reading a
product from the queue in an attempt to feed it to a
downstream site and the connection to that downstream site
goes down. A few of these messages now and then do not
indicate a problem, You can see more about this at
http://www.unidata.ucar.edu/cgi-bin/mfs/65/3769?354#mfs.
Again, this topic shows up regularly in the support
archives, probably because the log entry is so common.
Neither of these issues involve the product queue, so I've
deleted the pqmon output from this message. (Cleaner for
the support archives).
Also, in your earlier message you said you were running
V5.1.2. Are you aware that, in general, you don't need to
run pqexpire with 5.1.2? Unless there is a specific reason
to get rid of products in your queue before the space would
otherwise be needed, I would recommend against running
pqexpire. It can be a bottleneck. Maybe you still have the
line "exec pqexpire" in your ldmd.conf. If so you can
comment that out, then stop and restart the LDM.
Anne
--
***************************************************
Anne Wilson UCAR Unidata Program
address@hidden P.O. Box 3000
Boulder, CO 80307
----------------------------------------------------
Unidata WWW server http://www.unidata.ucar.edu/
****************************************************
> Dec 1 16:01:29 alice scour: exiting ...
> Dec 01 16:05:10 alice pqexpire[6391]: > Recycled 598823.804 kb/hr (
> 25084.588 prods per hour)
> Dec 01 16:09:39 alice is2(feed)[26884]: h_clnt_call: is2.wwb.noaa.gov:
> 1: time elapsed 31.827017
> Dec 01 16:09:39 alice as2(feed)[26886]: h_clnt_call: as2.wwb.noaa.gov:
> 1: time elapsed 31.864187
> Dec 01 16:09:43 alice pqact[6393]: pbuf_flush 4: time elapsed 2.892348
>
> Dec 01 16:10:10 alice pqexpire[6391]: > Recycled 598297.755 kb/hr (
> 25172.258 prods per hour)
> Dec 01 16:15:10 alice pqexpire[6391]: > Recycled 580766.803 kb/hr (
> 24751.753 prods per hour)
> Dec 01 16:17:32 alice DCMOSA[5283]: Starting Up
> Dec 01 16:17:38 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.155136
> Dec 01 16:19:26 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.517341
> Dec 01 16:19:28 alice DCFFG[27739]: Normal Termination
> Dec 01 16:19:28 alice DCFFG[27739]: Shutting Down
> Dec 01 16:19:44 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.546523
> Dec 01 16:19:56 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.453383
> Dec 01 16:20:02 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.067404
> Dec 01 16:20:05 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.955719
> Dec 01 16:20:11 alice pqexpire[6391]: > Recycled 569028.548 kb/hr (
> 24961.394 prods per hour)
> Dec 01 16:20:11 alice pqact[6393]: pbuf_flush 20: time elapsed
> 5.808064
> Dec 01 16:20:16 alice pqact[6393]: pbuf_flush 20: time elapsed
> 4.043067
> Dec 01 16:20:53 alice pqact[6393]: pbuf_flush 20: time elapsed
> 4.428729
> Dec 01 16:20:57 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.940115
> Dec 01 16:21:00 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.054513
> Dec 01 16:21:03 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.058657
> Dec 01 16:21:06 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.485180
> Dec 01 16:21:10 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.567991
> Dec 01 16:21:14 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.379963
> Dec 01 16:21:16 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.128571
> Dec 01 16:21:19 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.279898
> Dec 01 16:21:23 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.633631
> Dec 01 16:21:26 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.579510
> Dec 01 16:21:31 alice pqact[6393]: pbuf_flush 20: time elapsed
> 4.486743
> Dec 01 16:21:35 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.328678
> Dec 01 16:21:38 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.741416
> Dec 01 16:21:41 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.981837
> Dec 01 16:21:44 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.877144
> Dec 01 16:21:50 alice pqact[6393]: pbuf_flush 20: time elapsed
> 5.944428
> Dec 01 16:21:53 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.854860
> Dec 01 16:21:57 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.418834
> Dec 01 16:21:59 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.806684
> Dec 01 16:22:03 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.746676
> Dec 01 16:22:05 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.101540
> Dec 01 16:22:08 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.177798
> Dec 01 16:22:11 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.906679
> Dec 01 16:22:15 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.305279
> Dec 01 16:22:17 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.810623
> Dec 01 16:22:23 alice pqact[6393]: pbuf_flush 20: time elapsed
> 5.560840
> Dec 01 16:22:30 alice pqact[6393]: pbuf_flush 20: time elapsed
> 6.019310
> Dec 01 16:22:33 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.239494
> Dec 01 16:22:36 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.612594
> Dec 01 16:22:42 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.221680
> Dec 01 16:22:47 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.345156
> Dec 01 16:22:49 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.038491
> Dec 01 16:22:55 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.783552
> Dec 01 16:22:57 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.273048
> Dec 01 16:23:01 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.887906
> Dec 01 16:23:04 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.389434
> Dec 01 16:23:06 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.025302
> Dec 01 16:23:09 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.094248
> Dec 01 16:23:11 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.351357
> Dec 01 16:23:13 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.097404
> Dec 01 16:23:17 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.863524
> Dec 01 16:23:29 alice pqact[6393]: pbuf_flush 4: time elapsed 2.718058
>
> Dec 01 16:23:34 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.048115
> Dec 01 16:23:43 alice pqact[6393]: pbuf_flush 20: time elapsed
> 8.668860
> Dec 01 16:23:51 alice pqact[6393]: pbuf_flush 20: time elapsed
> 6.349345
> Dec 01 16:23:54 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.296253
> Dec 01 16:23:57 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.870694
> Dec 01 16:24:02 alice pqact[6393]: pbuf_flush 20: time elapsed
> 4.502862
> Dec 01 16:24:06 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.368944
> Dec 01 16:24:10 alice pqact[6393]: pbuf_flush 20: time elapsed
> 4.417863
> Dec 01 16:24:14 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.967715
> Dec 01 16:24:17 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.749150
> Dec 01 16:24:20 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.090045
> Dec 01 16:24:23 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.185266
> Dec 01 16:24:26 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.667035
> Dec 01 16:24:28 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.260759
> Dec 01 16:24:31 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.092945
> Dec 01 16:24:33 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.124764
> Dec 01 16:24:36 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.166947
> Dec 01 16:24:38 alice DCMSFC[28860]: Normal Termination
> Dec 01 16:24:39 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.655995
> Dec 01 16:24:42 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.590812
> Dec 01 16:24:45 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.791761
> Dec 01 16:24:45 alice DCMSFC[7221]: Normal Termination
> Dec 01 16:24:48 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.784941
> Dec 01 16:24:48 alice DCUAIR[7218]: Normal Termination
> Dec 01 16:24:51 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.025181
> Dec 01 16:24:54 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.737293
> Dec 01 16:24:56 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.566680
> Dec 01 16:24:57 alice DCACFT[7211]: Normal Termination
> Dec 01 16:24:57 alice DCLSFC[6641]: Normal Termination
> Dec 01 16:25:00 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.356381
> Dec 01 16:25:07 alice pqact[6393]: pbuf_flush 20: time elapsed
> 7.106521
> Dec 01 16:25:12 alice pqexpire[6391]: > Recycled 590898.399 kb/hr (
> 25368.633 prods per hour)
> Dec 01 16:25:12 alice pqact[6393]: pbuf_flush 20: time elapsed
> 4.270111
> Dec 01 16:25:16 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.649391
> Dec 01 16:25:20 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.858487
> Dec 01 16:25:25 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.931335
> Dec 01 16:25:29 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.974186
> Dec 01 16:25:31 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.057875
> Dec 01 16:25:33 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.246556
> Dec 01 16:25:48 alice pqact[6393]: pbuf_flush 20: time elapsed
> 3.198572
> Dec 01 16:25:51 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.597855
> Dec 01 16:25:51 alice DCPROF[8563]: Starting Up
> Dec 01 16:25:55 alice pqact[6393]: pbuf_flush 20: time elapsed
> 4.423134
> Dec 01 16:25:58 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.637231
> Dec 01 16:26:00 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.003642
> Dec 01 16:26:02 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.625878
> Dec 01 16:26:07 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.459529
> Dec 01 16:26:16 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.550718
> Dec 01 16:26:23 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.560152
> Dec 01 16:26:32 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.144738
> Dec 01 16:26:34 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.039939
> Dec 01 16:26:37 alice pqact[6393]: pbuf_flush 20: time elapsed
> 2.457398
> Dec 01 16:26:41 alice DCLSFC[9246]: Starting Up
> Dec 01 16:26:41 alice DCACFT[9247]: Starting Up
> Dec 01 16:26:42 alice DCPROF[8563]: Normal Termination
> Dec 01 16:26:42 alice DCAMOS[9252]: Starting Up
> Dec 01 16:26:45 alice DCMSFC[9261]: Starting Up
> Dec 01 16:27:00 alice DCUAIR[9402]: Starting Up
> Dec 01 16:30:12 alice pqexpire[6391]: > Recycled 584816.213 kb/hr (
> 25835.167 prods per hour)
> Dec 01 16:35:12 alice pqexpire[6391]: > Recycled 570200.493 kb/hr (
> 25769.357 prods per hour)
> Dec 01 16:36:04 alice DCFFG[12032]: Starting Up
> Dec 01 16:36:45 alice DCAMOS[9252]: Normal Termination
> Dec 01 16:36:51 alice DCMOSA[5283]: Normal Termination
> Dec 01 16:39:13 alice pqact[6393]: pbuf_flush 4: time elapsed 2.639967
>
> Dec 01 16:39:28 alice pqact[6393]: pbuf_flush 4: time elapsed 2.168347
>
> Dec 01 16:40:02 alice pqact[6393]: pbuf_flush 4: time elapsed 3.507078
>
> Dec 01 16:40:05 alice pqact[6393]: pbuf_flush 4: time elapsed 2.775522
>
> Dec 01 16:40:13 alice pqexpire[6391]: > Recycled 590540.188 kb/hr (
> 26005.975 prods per hour)
> Dec 01 16:41:13 alice is2(feed)[26884]: SRUS56 KSEW 011639 COR /pRR2SEA:
> RPC_TIMED_OUT (5)
> Dec 01 16:41:13 alice as2(feed)[26886]: SRUS56 KSEW 011639 COR /pRR2SEA:
> RPC_TIMED_OUT (5)
> Dec 01 16:41:13 alice as2(feed)[26886]: pq_sequence failed: I/O error
> (errno = 5)
> Dec 01 16:41:20 alice rpc.ldmd[6382]: child 26886 exited with status 1
> Dec 01 16:41:47 alice is2[13298]: Connection from is2.wwb.noaa.gov
> Dec 01 16:41:47 alice is2(feed)[13298]: Starting Up: 20001201164013.676
> TS_ENDT {{WMO|SPARE, "(^[A-FK-OQRU-X])|(^S[^D])"}}
> Dec 01 16:41:47 alice is2(feed)[13298]: topo: is2.wwb.noaa.gov
> WMO|SPARE
> Dec 01 16:42:15 alice as2[13371]: Connection from as2.wwb.noaa.gov
> Dec 01 16:42:15 alice as2(feed)[13371]: Starting Up: 20001201164013.676
> TS_ENDT {{WMO|SPARE, "(^[A-FK-OQRU-X])|(^S[^D])"}}
> Dec 01 16:42:15 alice as2(feed)[13371]: topo: as2.wwb.noaa.gov
> WMO|SPARE
> Dec 01 16:45:13 alice pqexpire[6391]: > Recycled 578997.480 kb/hr (
> 25811.269 prods per hour)
> Dec 01 16:46:33 alice DCFFG[12032]: Normal Termination
> Dec 01 16:46:59 alice DCUAIR[9402]: Normal Termination
> Dec 01 16:50:14 alice pqexpire[6391]: > Recycled 567987.723 kb/hr (
> 25765.970 prods per hour)
> Dec 01 16:53:14 alice pqact[6393]: pbuf_flush 4: time elapsed 2.101476
>
> Dec 01 16:53:49 alice pqact[6393]: pbuf_flush 4: time elapsed 2.117160
>
> Dec 01 16:55:14 alice pqexpire[6391]: > Recycled 585623.104 kb/hr (
> 25870.392 prods per hour)
> Dec 01 17:00:14 alice pqexpire[6391]: > Recycled 580853.332 kb/hr (
> 25906.743 prods per hour)
> Dec 01 17:00:56 alice DCUAIR[16665]: Starting Up
> Dec 01 17:05:15 alice pqexpire[6391]: > Recycled 570307.911 kb/hr (
> 25656.159 prods per hour)
> Dec 01 17:09:03 alice pqact[6393]: pbuf_flush 4: time elapsed 2.623771
>
> Dec 01 17:10:15 alice pqexpire[6391]: > Recycled 585587.472 kb/hr (
> 25409.914 prods per hour)
> Dec 01 17:15:15 alice pqexpire[6391]: > Recycled 577250.580 kb/hr (
> 25267.797 prods per hour)
> Dec 01 17:16:17 alice DCFFG[20260]: Starting Up
> Dec 01 17:20:16 alice pqexpire[6391]: > Recycled 567726.911 kb/hr (
> 25162.872 prods per hour)
> Dec 01 17:20:16 alice DCUAIR[16665]: Normal Termination
> Dec 01 17:22:53 alice DCUAIR[24356]: Starting Up
> Dec 01 17:23:27 alice pqact[6393]: pbuf_flush 4: time elapsed 2.216035
>
> Dec 01 17:25:16 alice pqexpire[6391]: > Recycled 580813.200 kb/hr (
> 25001.021 prods per hour)
> Dec 01 17:26:46 alice DCPROF[25017]: Starting Up
> Dec 01 17:27:26 alice DCSCD[25083]: Starting Up
>
> ...
> Dec 01 17:40:17 alice pqexpire[6391]: > Recycled 574536.045 kb/hr (
> 24343.064 prods per hour)
> Dec 01 17:41:43 alice DCSCD[27165]: Starting Up
> Dec 01 17:43:07 alice DCUAIR[24356]: Normal Termination
> Dec 01 17:45:17 alice pqexpire[6391]: > Recycled 565760.431 kb/hr (
> 24080.018 prods per hour)
> Dec 01 17:45:24 alice DCUAIR[27708]: Starting Up
> Dec 01 17:50:17 alice pqexpire[6391]: > Recycled 557106.267 kb/hr (
> 23869.878 prods per hour)
> Dec 01 17:55:17 alice pqexpire[6391]: > Recycled 567922.859 kb/hr (
> 23711.829 prods per hour)
> Dec 01 17:57:39 alice pqact[6393]: pbuf_flush 4: time elapsed 4.235711
>
> Dec 01 17:57:48 alice pqact[6393]: pbuf_flush 4: time elapsed 8.730530
>
> Dec 01 18:00:18 alice pqexpire[6391]: > Recycled 561976.161 kb/hr (
> 23589.844 prods per hour)
> Dec 01 18:05:18 alice pqexpire[6391]: > Recycled 554081.072 kb/hr (
> 23413.596 prods per hour)
> Dec 01 18:07:15 alice is2(feed)[13298]: SRUS54 KTSA 011806 /pRR5TUL:
> RPC_CANT_RECV (4)
> Dec 01 18:07:21 alice rpc.ldmd[6382]: child 13298 exited with status 1
> Dec 01 18:07:28 alice as2(feed)[13371]: SFUS41 KWBC 011805:
> RPC_CANT_RECV (4)
> Dec 01 18:07:28 alice as2(feed)[13371]: pq_sequence failed: I/O error
> (errno = 5)
> Dec 01 18:07:28 alice as2(feed)[13371]: Exiting
> Dec 01 18:07:34 alice rpc.ldmd[6382]: child 13371 exited with status 1
> Dec 01 18:07:35 alice is2[419]: Connection from is2.wwb.noaa.gov
> Dec 01 18:07:35 alice is2(feed)[419]: Starting Up: 20001201180711.681
> TS_ENDT {{WMO|SPARE, "(^[A-FK-OQRU-X])|(^S[^D])|(^Y[HO]XX84)"}}
> Dec 01 18:07:35 alice is2(feed)[419]: topo: is2.wwb.noaa.gov WMO|SPARE
> Dec 01 18:07:50 alice as2[435]: Connection from as2.wwb.noaa.gov
> Dec 01 18:07:50 alice as2(feed)[435]: Starting Up: 20001201180724.923
> TS_ENDT {{WMO|SPARE, "(^[A-FK-OQRU-X])|(^S[^D])|(^Y[HO]XX84)"}}
> Dec 01 18:10:18 alice pqexpire[6391]: > Recycled 563406.563 kb/hr (
> 23290.191 prods per hour)
> Dec 01 18:15:19 alice pqexpire[6391]: > Recycled 555397.786 kb/hr (
> 23121.883 prods per hour)
> Dec 01 18:20:19 alice pqexpire[6391]: > Recycled 546792.063 kb/hr (
> 23068.774 prods per hour)
> Dec 01 18:24:45 alice DCPROF[6828]: Starting Up
> Dec 01 18:25:19 alice pqexpire[6391]: > Recycled 545014.124 kb/hr (
> 22968.134 prods per hour)
>
>
> ------- End of Forwarded Message
