2012 Unidata NetCDF Workshop > Best Practices for Data providers
20.9 Missing Data Values
To indicate that data values are missing, invalid, or not written,
special values are conventionally used.
NetCDF data may include variable values
that are not written, either on purpose or unintentionally.
To detect attempts to later read unwritten data,
the netCDF library initializes data with either
- the value of a variable's
_FillValue
attribute,
if it exists, or else
-
a type-specific default fill value, defined as part of the
netCDF interface, or
-
nothing, leaving the value undetermined, if the writer sets "no
fill" mode.
Recommendations
- Use the default netCDF fill values for each type or specify a
fill value with the _FillValue attribute.
- For floating-point data, consider using an IEEE
NaN (Not a Number) value, because any computation
using a NaN results in a NaN. Detection of NaNs requires special care,
for example use of isnan() function.
- Alternatively or in addition, set the valid_range attribute for each
variable that uses missing values, and make sure all valid data is within
that range, and all missing or invalid data is outside of that range. Again,
the client software must recognize and make use of this information. Example:
variables:
float rel_humidity( z, y, x);
rel_humidity:units = "percent";
rel_humidity:valid_range = 0.0f, 1.0f;
- The
_FillValue
attribute, if used, should have the
same data type as the variable
it describes. If the variable is packed, _FillValue
should use
the packed type.
- Turning on "no fill" mode can make writes faster, but this may
be an unwise optimization unless you know you will write all data
values: it prevents later detection of unwritten data.
2012 Unidata NetCDF Workshop > Best Practices for Data providers