Re: [netcdf-java] Date precision while aggregating data

I think you've all more or less pointed to the fact that storing millisecs in a 
long is a good solution (which both java.util.Date and the joda library also 
decided on).

Rich you are seeing that CDM/NcML is just fine with longs, even though they 
cant be stored in netcdf-3 (they are ok in netcdf-4).

Anyway, I think one improvement is to make sure behind the scenes im using 
millisecs in a long instead of seconds in doubles. Im pretty sure that will 
solve the roundoff problem. Im hoping it will be easy enough to put in both 2.2 
and 4.0.

I will probably also start to use joda, though that is still TBD, and wont 
affect the API for now. I will also support ISO dates in CDM/NcML, in addition 
to udunit dates. Any of that new developement will be in 4.0 only.

thanks for everyone's thoughts about this....

Rich Signell wrote:
Bob,

I like this solution.    While I would like to use units of days and
have rounding to milliseconds in NetCDF Java, that might not appeal to
someone who wanted even more accuracy.   And since both our existing
USGS database and Java have fundamental units of milliseconds, it
makes sense to use milliseconds for our units.

So my original NcML looked like:

<variable name="time" shape="time" type="double">
  <attribute name="units" value="days since 1858-11-17 00:00:00 UTC"/>
  <values start="47865.7916666665110000" increment="0.0416666666666667"/>
</variable>

While the NEW one looks like:

<variable name="time" shape="time" type="double">
  <attribute name="units" value="milliseconds since 1858-11-17 00:00:00 UTC"/>
  <values start="4154364000000" increment="3600000"/>
</variable>

This NEW form for the NcML returns the desired "05-Dec-1989 19:00:00"
as the start time.

And IMHO, it's even a bit "more readable"!    ;-)

Thanks,
Rich




On Fri, May 16, 2008 at 12:01 PM, Bob Simons <Bob.Simons@xxxxxxxx> wrote:
While one can debate whether the netCDF libraries should round (not
truncate) date/time values stored as doubles to the nearest millisecond
(I think they should), or whether it's a bug in Java (I think not - Java
does date/time calculations with longs), it doesn't solve the immediate
problem of storing date/time values in netCDF files for use with the
current netCDF libraries.

I think there is a simple solution: store date/time values with units
that allow all of the date/time values to be expressed as integer values
(in the mathematical sense). In practice, the values often will need to
be calculated as Java longs and stored as Java doubles in netCDF files
to gain extra precision, since there is no support for longs in netCDF
files.

For example, use "seconds since 1970-01-01T00:00:00Z" if your dates/time
data is measure to the nearest second.  Or, use milliseconds, minutes,
hours, or days, ... as appropriate.  Even when millisecond values are
stored as doubles and even if the "since" time is in the 1800's, there
should be no round-off error and Java and netCDF-java should calculate
the dates/times correctly.


Sincerely,

Bob Simons
Satellite Data Product Manager
Environmental Research Division
NOAA Southwest Fisheries Science Center
1352 Lighthouse Ave
Pacific Grove, CA 93950-2079
(831)658-3205
bob.simons@xxxxxxxx

The contents of this message are mine personally and
do not necessarily reflect any position of the
Government or the National Oceanic and Atmospheric
Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <><
_______________________________________________
netcdf-java mailing list
netcdf-java@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: 
http://www.unidata.ucar.edu/mailing_lists/






  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: