[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20000229: request for sizes of GOES imagery files (cont.)



>From: Unidata User Support <address@hidden>
>Organization: Unidata Program Center
>Keywords: GVAR imager sounder AREA files

Chad,

>I have a few questions to help clarify the information you are
>requesting. I'm assuming you are interested in numbers for the rawest
>and highest resolution (temporal and spatial) data available, but just
>wanted to clarify.

Yes, that is correct.  What I need to do is present a picture of just
what is available to the User Committee during their next meeting
(March 16/17).  My objective was to be able to say something like here
is all that data that is available from SSEC; here is the expected
compression we can get using PNG; and here are the likely sizes and
frequency of IDD transmitted products.  I did some quick figuring for
products that are already in the UW stream as follows and bounced
them off of a user a couple of weeks ago:

  Our feeling has been that the great majority of sites are having
  problems getting the images in their current sizes.  To put some
  numbers on these products, a current VIS GOES East of West image in
  the broadcast is anywhere from 1.2 to 1.8 MB (assuming that it is not
  dark; those images compress really well).  Its uncompressed size is
  on the order of 2.3 MB.  From these numbers, you can see that we are
  only getting a compression ratio of 75%.  Some experimentation at the
  UPC by Steve Chiswell has shown that use of PNG compression can give
  us much better compression numbers; broadcast products on the order
  of 35-40% of their original size.

  Even with these savings, this would translate into VIS products of
  the following size:

  resolution   product size   Comments
  ------------+--------------+---------
  4 km         840 KB         current resolution
  2 km         3.3 MB         double LINe and ELEment resolution
  1 km        13.3 MB         quadruple LINe and ELEment resolution

  Of course, this is the worst case scenario.  IR imagery is already
  being sent out at its maximum resolution, and it compresses better
  than VIS imagery.

These comments relate to images that have been blown down preferrentially
by a factor of two in the element dimension (i.e., element oversampling
has been removed by blowdown).

My question to the user was then what he would believe to be exceptable
and desirable.  I got him to admit that increased temporal resolution
for the current products was marginally more important than sending
out the higher resolution VIS images.

>Do you request information for all bands (1-5),

Yes.

>all coverages (CONUS, FD, etc),

Yes.

>and full spatial and bit resolutions?

Yes.  I can then work from there to reduce the numbers (i.e., 10 bit -> 8 bit;
element oversampled -> blown down; etc.).

>Is this for raw sounder products (all 19 bands) from the satellite as
>apposed to sounder derived products generated by CIMSS?

My email was about the raw sounder products (all 19 bands).  The CIMSS
products are also improtant since Steve Ackerman is very keen to see
them distributed by the IDD to Unidata sites.  If you had numbers for
those products, I would appreciate you passing them on as well.

>Again, do you mean the temporal and spatial resolution of products
>available direct from the satellite?

Yes.  My thinking was that somewhere at SSEC there must be some numbers
on what products are received from the GVAR platforms; how large they
are; and how frequently they come in.

>Would this be for 1-byte or 2-byte data (of course the math is easy
>regardless of the numbers we provide).

For 2-byte data since, as you note, I can easily divide by 2 to get 
numbers for 1-byte data.  Again, I am only looking for ballpark numbers.
I am not worried about too fine a detail since I will have to apply
the PNG compression factor to them anyway, and those numbers are highly
variable depending on image band (IR compresses much better than VIS
as you might expect).

>We'll get the information to you ASAP.

Thanks Chad.  I really appreciate the help!

On another, less important topic: did you get my email about the
existence of stand alone surface, raob, etc. decoders?  The hoops I had
to jump through to decode some old, raw data using XCD were pretty
ugly.  I had to modify ingetext.pgm and dmraob.pgm to add DAY= keywords
so that the DAY included in the decoded files was correct.  The effort
paid off as I was able to decode the data successfully.  I may even
leave in the DAY= support in ingetext.pgm and dmraob.pgm; they would be
hidden "features".

Thanks again.

Tom

>From address@hidden  Tue Feb 29 14:20:45 2000

> >We'll get the information to you ASAP.
> Thanks Chad.  I really appreciate the help!

OK. That's what I figured, but just wanted to make sure.

> On another, less important topic: did you get my email about the
> existence of stand alone surface, raob, etc. decoders? 

Yes I did, but I was unexpectedly out of town all of last week and I'm
still playing catchup.

> The hoops I had
> to jump through to decode some old, raw data using XCD were pretty
> ugly.  I had to modify ingetext.pgm and dmraob.pgm to add DAY= keywords
> so that the DAY included in the decoded files was correct.  The effort
> paid off as I was able to decode the data successfully.  I may even
> leave in the DAY= support in ingetext.pgm and dmraob.pgm; they would be
> hidden "features".

I have not made stand alone decoders to decode old raw data. A year or
so ago (maybe longer) we talked about this effort to support our
conventional data archive. Bug fixes are made to decoders constantly,
but our archive contains the decoded observations. Running the data
through the decoder again when it is retrieved is a better solution to
providing accurate data to the user. Unfortunately we never did the
work.

You are correct though. The only problem with recoding old raw data with
the -XCD decoders is setting the correct date. If you wish to run the
decoder stand-alone, there is another problem - the data monitors won't
start unless activated with DECINFO. 

-Chad

--
Chad W. Johnson
Computer Programmer/Meteorologist
Space Science and Engineering Center
University of Wisconsin - Madison
1225 W. Dayton Street
Madison, WI 53706

E-mail: address@hidden
Phone:  (608) 265-5292
Fax:    (608) 263-6738