And, here's some more examples, where it is extremely data dependent...
It's also significantly slower.
ETA grib file bzip2 gzip
12.208MB 6.97MB 8.78MB
24.6seconds 6.7seconds
MMOUT D1 file bzip2 gzip
266.7MB 187.4MB 180.3MB
9.0 minutes 2.5 minutes (!)
Laurie Carson
On 2000.08.09, Phil Rasch wrote:
:
: The compression gains must be extremely data dependent.
:
: Here are the results for the compression of a dataset that consists of
: 32bit floats of a collection of many meteorological fields from the NCAR CCM3:
:
: 35577060 bytes ENSANNAVG-0105.nc
: 26739972 bytes ENSANNAVG-0105.nc.bz2
:
: The compression is very small.
:
: Phil
:
: On Wed, Aug 09, 2000 at 08:03:50AM -0600, Roy Mendelssohn wrote:
: > There was some question as to how well Bzip2 would compress floating
: > point data. We just did a test on a large NetCDF file, containing a
: > years worth of global, 1-degree six-hourly pressure fields, so that
: > the data is floating point. here are the sizes from a Unix ls
: > command (i.e in 512 byte units).
: >
: > raw.nc 380,551,616
: >
: > raw.nc.Z 143,716,211
: >
: > raw.nc.bz2 55,633,062
: >
: >
: >
: > Bzip2 gave about the same 7:1 compression that Russ reported, and
: > roughly 3 times better that Unix compress.
: >
:
: --
: ----------------------------------------
: Phil Rasch, Climate Modeling Section, National Center for Atmospheric Research
: P.O. Box 3000, Boulder CO 80307
: internet: pjr@xxxxxxxx, http://www.cgd.ucar.edu/cms/pjr
: Phone: 303-497-1368, FAX: 303-497-1324
