On Fri, Jul 8, 2016 at 1:26 PM, Ismail SEZEN <sezenismail@xxxxxxxxx> wrote:
> The answer from ESRL PSD data about precision and least_significant_digit:
>
> Hi
> Least significant digit is in the original grib files.
OK, so despite the docs these mean two different things -- Least
significant digit seems to represent the actual precision of the original
values (maybe -- it could have a specific meaning in GRIB...)
> Precision is calculated here and is related to the packing. The precision
> may be a little sloppy with regards to our packing.
>
and precision is related to the packing.
Hopefully, they have used a packing scheme that handles the full precision
of the original data, so you could jsut ignore that and use least
significant digit to guide your rounding.
> We hope to be transitioning to netCDF4 at some point soon, which is what
> we use for other datasets. It does not need scale/offset packing as it
> packes internally.
>
that would be a nice step.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@xxxxxxxx
