[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20000831: pnga2area timeouts reading 4.7 MB compressed image

>From: Pepo Juega <address@hidden>
>Organization: Instituto Nacional de Meteorologia
>Keywords: 200008311046.e7VAk3N24792 ldm-mcidas pnga2area

Pepo,

>I finally reached one of my limits when handling GOES GVAR images.
>
>At area2png[]: 
>Error mallocing 453404688 bytes for compressed output file
>
>This happens in my SDI ingestor, a clone PC with 64MB of RAM and
>about 216MB of swap area for Solaris. Sounds logical.

Sounds logical to me also.

>But I can handle smaller AREAs for example:
>
>Aug 31 09:18:29 area2png[19501]: Starting Up
>input AREA size: 105419756
>Aug 31 09:18:29 area2png[22532]: output file pathname: GOES8IRFD...
>image shape: 2503 x 5212
>image line length: 42116
>deflated length 4696473
>Aug 31 09:21:00 area2png[22532]: doPNG:: 105419756   4696473  0.0446
>Aug 31 09:21:11 inggoes pqinsert

This is pretty good compression!

>when it reaches the other end, ...
>
>Aug 31 09:21:47 pnga2area[11347]: Starting Up
>Aug 31 09:22:47 pnga2area[11347]: pnga2area: no data read in 60 seconds,
>exiting
>Aug 31 09:22:47 goes data/AREA/REC/GOES8IRFD.1_20000831.0915 AREA8130
>Aug 31 09:22:48 goes IMGCOPY LDM/GOES8IRFD.1 LDM/GOES8_IR_FD
>Beginning Image Data transfer, bytes= 105419756
>Transferring AREA data outbound, bytes= 105419836
>IMGCOPY: LDM/GOES8IRFD.1 copied to LDM/GOES8_IR_FD.4
>imgcopy.k: done

This makes me think that the downstream machine is slow enough that the
image doesn't get fully uncompressed in one minute.  This timeout value
can be changed pretty easily in pnga2area.c:

unsigned         alarm_timeout = 60; /* seconds; should suffice */

You could try changing 'alarm_timeout' to something like 120, 180,
etc. to see if the timeout indication goes away.

Another possibility is that pnga2area is not getting an EOF indication
from the LDM's pqact routine.  This would account for the image 
apparently being correctly decoded.

Still a third possibility is that you do not have a '-close' flag
on the pnga2area invocation in your pqact.conf file.  Here is an
example:

MCIDAS  ^pnga2area .*
        PIPE    -close
        pnga2area ...

The '...' simply represents the rest of the pnga2area command line.

Does your pqact.conf action have a '-close' flag for PIPE?  If not,
I would add one and then send a HUP signal to pqact.

>...so, the image gets copied all right even with that error
>message about "no data read in 60 seconds, exiting"
>This does not happen all the time. Other times I get:
>
>Aug 31 08:58:43 pnga2area[10908]: Starting Up
>Aug 31 08:59:42 pnga2area[10908]: unPNG:: 12892728  113927188  8.8365
>Aug 31 08:59:42 pnga2area[10908]: Exiting
>Aug 31 08:59:42 goes data/AREA/REC/GOES8IRFD.1_20000831.0845 AREA8130
>Aug 31 08:59:42 goes IMGCOPY LDM/GOES8IRFD.1 LDM/GOES8_IR_FD
>Beginning Image Data transfer, bytes= 113927188
>Transferring AREA data outbound, bytes= 113927268
>IMGCOPY: LDM/GOES8IRFD.1 copied to LDM/GOES8_IR_FD.3
>
>My concern is if the error message is serious or just a warning.
>Any comments?

This really does sound like you don't have the '-close' flag on the
PIPE action.  Please let me know if this is not true.

Tom