Joe McLean wrote:
Hi John,
Attached is an NcML aggregation file which is a good example of NDBC time
series data which has missing values.
The netcdf elements with missing data are below.
<!-- 2002-10 has 0 hrs = 0 days -->
<netcdf
uri='dods://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/burl1_2002_10_stdmet.nc'/>
MISSING 744 of 744 hrs
I could create a dummy file using all missing_value values and plug it in place
of this uri.
<!-- 2003-04 has 184 hrs = 7.66666666666667 days -->
<netcdf
uri='dods://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/burl1_2003_04_stdmet.nc'/>
MISSING 536 of 720 hrs: 'Apr-08 16:00' thru 'Apr-30 23:00'
<!-- 2003-05 has 414 hrs = 17.25 days -->
<netcdf
uri='dods://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/burl1_2003_05_stdmet.nc'/>
MISSING 330 of 744 hrs: 'May-01 00:00` thru 'May-14 17:00'
The last two are missing data on either end of the expected time axis. In the
previous example (email 22SEP 16:50), the missing data was in the middle of the
expected time axis. I'll leave this to you to see how one could use NCML to
aggregate such files, but using agg type 'joinNew' and adding a range attribute
to the netcdf element (then, when the missing data is in the middle of the time
axis, the uri could be listed twice) might be an idea. Then there is the whole
issue of irregularly stepped time axes - I realize with NCML aggregations, one
can hard code the coordinate values, but these aggregations are 17,000+ time
steps - what issues would be involved then?.
Joe
i have an experimental version that allows unit conversion, as long as
the individual files have correct udunits.
get a new version of the AS:
ftp://ftp.unidata.ucar.edu/pub/thredds/dodsCupdate.war
and try this in your NcML, instead of the <values> element:
<variable name='time' type='double' shape='time'>
<attribute name='long_name' type='string' value='time coordinate'/>
<attribute name='units' type='string' value='hours since 2003-08-19'/>
<units convertTo="hours since 2003-08-19" />
</variable>
where you change your units (in both places) to whatever you want.
let me know how that works.
CAUTION: the syntax is experimental, although i will try to keep the
functionality. so test it on one before you do a lot of work converting,
unless it will be easy to change.
