-------- Original Message --------
Subject: Re: valid_min, valid_max, scaled, and missing values
Date: Tue, 27 Feb 2001 14:18:47 -0700
From: John Caron <caron@xxxxxxxxxxxxxxxx>
To: Russ Rew <russ@xxxxxxxxxxxxxxxx>
References: <200102272037.f1RKbuL27791@xxxxxxxxxxxxxxxx>
Russ Rew wrote:
>
> John,
>
> > What do you think about using that info in a "standard" implementation
> > of missing_data? This is a generalization of what Harvey has suggested,
> > I think: that the type of the missing_data attribute determines if its
> > packed or unpacked. He suggested (I think) that it is always the type of
> > the packed data.
>
> I assume you mean "missing_value" attribute rather than a new
> "missing_data" attribute?
yes, including valid_range, _FillValue etc.
>
> First, I think we should use the "_FillValue" attribute if it exists
> and there is no "missing_value" attribute. I would only use the
> "missing_value" attribute in case the "_FillValue" attribute is not
> defined.
what happens if they are both present?
>
> Currently I believe that the netCDF C library requires that the
> _FillValue attribute be of the same external type as the variable,
> regardless of whether the variable is packed or not (the C library
> ignores all packing attributes). If the variable is packed, this is
> the type of the packed data and similarly if the data is unpacked,
> this is the type of the unpacked data.
you mean packed, ie _FillValue is always the same type as data type?
> So I don't think it's possible
> to have the _FillValue use the unpacked units for a packed variable.
ok
>
> And I'm not sure this makes sense anyway, for example to have a
> low-precision floating-point variable packed as bytes with an unpacked
> _FillValue of -999, since -999 can't be represented as a byte. It
> seems as if the _FillValue has to be represented as packed in this
> case. But maybe I'm not understanding the question ...
>
> --Russ
Ok, let me start fresh. The issue is, what should be a "standard"
implementation of scale and offset, and missing data. Now missing data
can be specified using several standard attriutes: _FillValue,
missing_value, valid_range, and/or valid_min/max.
So heres a possible implementation:
1) _FillValue must have the same type of the data.
2) missing_value should have the same type of the data;
3) _FillValue and missing_data values are in the units of the external
data, which is the packed data if using scale/offset.
4) if valid_range is present, valid_min and valid_max attributes are
ignored. Otherwise, the valid_min and/or valid_max is used to construct
a valid range.
5) if valid_range has a wider type than the data, and there is a
scale/offset, then it is interpreted as using the same units as the
unpacked data. Otherwise valid_range is interpreted as using the same
units as the external (packed if applicable) data.
This attempts to be maximally compliant with our published atribute
conventions, and to deal with existing datasets that use valid_range in
unpacked units.
