Re: [netcdfgroup] Precision an dSignificant digits in netcdf

On Fri, Jul 8, 2016 at 1:26 PM, Ismail SEZEN <sezenismail@xxxxxxxxx> wrote:

> The answer from ESRL PSD data about precision and least_significant_digit:
>
> Hi
> Least significant digit is in the original grib files.


OK, so despite the docs these mean two different things -- Least
significant digit seems to represent the actual precision of the original
values (maybe -- it could have a specific meaning in GRIB...)


> Precision is calculated here and is related to the packing. The precision
> may be a little sloppy with regards to our packing.
>

and precision is related to the packing.

Hopefully, they have used a packing scheme that handles the full precision
of the original data, so you could jsut ignore that and use least
significant digit to guide your rounding.


> We hope to be transitioning to netCDF4 at some point soon, which is what
> we use for other datasets. It does not need scale/offset packing as it
> packes internally.
>

that would be a nice step.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@xxxxxxxx
  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: