We have thought a lot about packing in netCDF and considered various schemes on
how to do it. Variable length compression schemes would probably interfere with
efficient subsetting, but it seems to me that fixed bit size packing should be
doable. We have this on our wish list, i mean to-do list, for the next round of
netCDF development.
since COARDS is lat/lon only, you cant do arbitrary GRIB conversion. CF seems
like a good candidate, however. Has anyone investigated the feasibility of this?
gribtonc uses NUWG, and has various limitations. We are interested in possibly
upgrading gribtonc. If anyone can make use of more flexible GRIB to netCDF
conversion, I'd like to hear about your "use-case".
I think in order to correctly group the GRIB records into 3 or 4 dimensional
netCDF variables, you will need (for the general case) some sort of
configuration info for the converter, although I suppose a "common case" could
be assumed. Anyone have any thoughts on that?
Timothy Hume wrote:
Hi,
This discussion reminded me of how GRIB packs data. Ideally, it would be
nice for NetCDF to be able to handle data with an arbitrary number of
bits. Many meteorological data can be packed into only 9 or 10 bits (often
less), so packing them into 16 bit short integers is "wasteful". Aside
from that many satellite data are "naturally" 10 bit, and increasing them
to 16 bits can cause the file size to increase by tens of megabytes per
image.
By the way, does anyone know of software that can convert GRIB data to
COARDS or CF conventions? gribtonc converts GRIB to NUWG conventions?
Tim Hume
By the way, does anyone know of a GRIB to COARDS or CF
On Wed, 2 Oct 2002, Mark A Ohrenschall wrote:
Hello,
In the case of a packed variable (in which scale_factor and add_offset
are used) both the COARDS and CF conventions indicate that missing_value
and _FillValue should be likewise packed:
COARDS: "In cases where the data variable is packed via the scale_value
attribute this implies that the missing_value flag is likewise packed."
CF: "The missing values of a variable with scale_factor and/or
add_offset attributes (see section 8.1) are interpreted relative to the
variable's external values, i.e., the values stored in the netCDF file."
I'm assuming that for the sake of consistency, this means that all
statistical variable attributes should be packed as well, e.g.,
valid_range and actual_range, as well as mean and standard_deviation. Is
this true?
So for example, if I have real world data values for temperature between
-1.6 and 31.4 and I'm applying a scale_factor of 0.1 then I would say
the valid_range is -16, 314 and the mean is 116 (not 11.6)?
Thanks,
Mark
