Another conversion question for Russ

• To: netcdf-hdf@xxxxxxxxxxxxxxxx
• Subject: Another conversion question for Russ
• From: Quincey Koziol <koziol@xxxxxxxxxxxxx>
• Date: Thu, 23 Oct 2003 12:25:21 -0500 (CDT)

Hi Russ,
    When an integer value is converted to a float, but the floating-point
value doesn't have enough precision to hold the correct value, it gets rounded.
Is this an "overflow" from netcdf-3's perspective?  It will be an "overflow" in
the implementation we are adding.

    Quincey
>From owner-netcdf-hdf@xxxxxxxxxxxxxxxx 23 2003 Oct -0600 11:28:36 
Message-ID: <wrxn0br2797.fsf@xxxxxxxxxxxxxxxxxxxxxxx>
Date: 23 Oct 2003 11:28:36 -0600
From: Ed Hartnett <ed@xxxxxxxxxxxxxxxx>
In-Reply-To: <200310231718.h9NHIreb006353@xxxxxxxxxxxxxxxxxxxxxx>
To: Quincey Koziol <koziol@xxxxxxxxxxxxx>
Subject: Re: automatic type conversion issues: range errors
Received: (from majordo@localhost)
        by unidata.ucar.edu (UCAR/Unidata) id h9NHScDb018337
        for netcdf-hdf-out; Thu, 23 Oct 2003 11:28:38 -0600 (MDT)
Received: from rodney.unidata.ucar.edu (rodney.unidata.ucar.edu 
[128.117.140.88])
        by unidata.ucar.edu (UCAR/Unidata) with ESMTP id h9NHSbOb018332;
        Thu, 23 Oct 2003 11:28:37 -0600 (MDT)
Organization: UCAR/Unidata
Keywords: 200310231728.h9NHSbOb018332
Cc: netcdf-hdf@xxxxxxxxxxxxxxxx
References: <200310231718.h9NHIreb006353@xxxxxxxxxxxxxxxxxxxxxx>
Lines: 39
User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.2
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Sender: owner-netcdf-hdf@xxxxxxxxxxxxxxxx
Precedence: bulk

Quincey Koziol <koziol@xxxxxxxxxxxxx> writes:

> Hi Ed,
>     HDF5 provides the user with a method of catching these sort of errors,
> using the H5T{get|set}_overflow() routines.  The overflow routine is called
> any time that a value in the source can't be represented exactly in the
> destination, with the datatypes provided.  This allows a user application to
> catch range errors and potentially change the value set for the destination
> value.

Perfect! Thanks. I will start checking this.

>     Currently, there isn't a way to pass the information about the fill value
> along to the user's callback routine, which looks like it would be necessary 
> to
> fulfill the current netCDF-3 functionality.  Additionally, I think it would be
> better to have these routines set on an individual data transfer, instead of
> globally for the library, so I think we should move them to be get|set 
> routines
> on a data transfer property list.

OK.

>     For now, you can use the current H5Tset_overflow routine to trap any
> overflows and set them to be zero and when I've fixed this up better, you will
> have access to the actual fill value the user set for the dataset.
> 

So right now it will substitute a zero for a value that has
overflowed?

If one is storing temperature data, for example, zero is a perfectly
valid value, so that would be a real problem. The very extremes
(i.e. values which overflowed the expected type) will be plopped right
in the center of the data distribution, at zero.

I think that the use of the fill value is mandatory in this case. I
can't imagine a case where I would prefer overflowed values to be
zero.

Ed

>From owner-netcdf-hdf@xxxxxxxxxxxxxxxx 23 2003 Oct -0600 11:31:09 
Message-ID: <wrxismf274y.fsf@xxxxxxxxxxxxxxxxxxxxxxx>
Date: 23 Oct 2003 11:31:09 -0600
From: Ed Hartnett <ed@xxxxxxxxxxxxxxxx>
In-Reply-To: <200310231725.h9NHPLoG006427@xxxxxxxxxxxxxxxxxxxxxx>
To: Quincey Koziol <koziol@xxxxxxxxxxxxx>
Subject: Re: Another conversion question for Russ
Received: (from majordo@localhost)
        by unidata.ucar.edu (UCAR/Unidata) id h9NHVDTL022771
        for netcdf-hdf-out; Thu, 23 Oct 2003 11:31:13 -0600 (MDT)
Received: from rodney.unidata.ucar.edu (rodney.unidata.ucar.edu 
[128.117.140.88])
        by unidata.ucar.edu (UCAR/Unidata) with ESMTP id h9NHV9Ob022760;
        Thu, 23 Oct 2003 11:31:09 -0600 (MDT)
Organization: UCAR/Unidata
Keywords: 200310231731.h9NHV9Ob022760
Cc: netcdf-hdf@xxxxxxxxxxxxxxxx
References: <200310231725.h9NHPLoG006427@xxxxxxxxxxxxxxxxxxxxxx>
Lines: 18
User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.2
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Sender: owner-netcdf-hdf@xxxxxxxxxxxxxxxx
Precedence: bulk

Quincey Koziol <koziol@xxxxxxxxxxxxx> writes:

> Hi Russ,
>     When an integer value is converted to a float, but the floating-point
> value doesn't have enough precision to hold the correct value, it gets 
> rounded.
> Is this an "overflow" from netcdf-3's perspective?  It will be an "overflow" 
> in
> the implementation we are adding.
> 
>     Quincey

My understanding is that this is NOT an overflow for netcdf. Loss of
precision is not an error. Only values that are too large to convert
are range errors.

For example, I don't think netcdf will ever have a range problem
converting ints to floats. Russ, am I completely wrong about that?

Ed