Simon,
Thanks for the data. It helped reveal a problem with caching and
aggregations. This should be fixed in the new 4.5.5 release.
Please give a shot and let us know if you're still having problems.
Thanks,
Ryan
On Wed, Mar 4, 2015 at 11:12 AM, Ryan May <rmay@xxxxxxxx> wrote:
> Simon,
>
> That looks like it should work. Can you send me or point me to the actual
> data files so I can try to reproduce with your NCML?
>
> Ryan
>
> On Fri, Feb 27, 2015 at 9:12 AM, Wright, Simon M. <smw@xxxxxxxxx> wrote:
>
>> Dear All
>>
>>
>>
>> I’m trying to set up a nested aggregation to use in THREDDS 4.5.2 but am
>> having some issues, so wonder if anyone can help.
>>
>>
>>
>> I’m following the Nested Aggregation example at
>> http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/ncml/Aggregation.html
>>
>> and have created the following ncml file:
>>
>>
>>
>> <?xml version='1.0' encoding='UTF-8'?>
>>
>> <netcdf xmlns='http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2'>
>>
>> <aggregation dimName="time" type="joinExisting">
>>
>> <netcdf>
>>
>> <aggregation type="union">
>>
>> <netcdf
>> location="../detail/MORECSDailyRf19610101to20131231.nc" />
>>
>> <netcdf
>> location="../detail/MORECSDailySun19610101to20131231.nc" />
>>
>> <netcdf
>> location="../detail/MORECSDailyTemp19610101to20131231.nc" />
>>
>> <netcdf
>> location="../detail/MORECSDailyVapP19610101to20131231.nc" />
>>
>> <netcdf
>> location="../detail/MORECSDailyWSp19610101to20131231.nc" />
>>
>> </aggregation>
>>
>> </netcdf>
>>
>> <netcdf>
>>
>> <aggregation type="union">
>>
>> <netcdf
>> location="../detail/MORECSDailyRf20140101to20141231.nc" />
>>
>> <netcdf
>> location="../detail/MORECSDailySun20140101to20141231.nc" />
>>
>> <netcdf
>> location="../detail/MORECSDailyTemp20140101to20141231.nc" />
>>
>> <netcdf
>> location="../detail/MORECSDailyVapP20140101to20141231.nc" />
>>
>> <netcdf
>> location="../detail/MORECSDailyWSp20140101to20141231.nc" />
>>
>> </aggregation>
>>
>> </netcdf>
>>
>> </aggregation>
>>
>> </netcdf>
>>
>>
>>
>> The data in the netCDF files is daily data with time values being
>> relative to 01/01/1961. For the files suffixed with
>> 19610101to20131231.nc there are 19358 values on the time dimension with
>> values ranging between 0.0 and 19357.0. For those files suffixed
>> 20140101to20141231.nc there are 365 values on the time dimension with
>> values ranging between 19358.0 and 19722.0. All other dimensions in the
>> netCDF files – eastings and northings on the British National Grid- are the
>> same.
>>
>>
>>
>> When I check the NcML file using the ToolsUI version 4.5 using NcML and
>> Aggregation tabs I get the following error:
>>
>>
>>
>> java.nio.file.InvalidPathException: Illegal char <:> at index 4
>>
>>
>>
>> However, when I check the ncml file using the ToolsUI version 4.3 again
>> using the NcML and Aggregation tabs I see the aggregation that I expect
>> with 5 union-ed variables with a time dimension having 19358 + 365 = 19723
>> values. The following summary is displayed in the ToolsUI 4.3 interface:
>>
>>
>>
>> Type=joinExisting
>>
>> dimName=time
>>
>> Datasets
>>
>>
>> file:F:/THREDDS/MORECS/20150227/aggregation/DetailWholeDataset.ncml#4138984
>> range=[0:19358) (19358)
>>
>>
>> file:F:/THREDDS/MORECS/20150227/aggregation/DetailWholeDataset.ncml#25192620
>> range=[19358:19723) (365)
>>
>> timeUnitsChange=false
>>
>> totalCoords=19723
>>
>>
>>
>> Aggregation Variables
>>
>> time(time=19723)
>>
>> rainfall_amount(time=19723, y=28, x=17)
>>
>> duration_of_sunshine(time=19723, y=28, x=17)
>>
>> air_temperature(time=19723, y=28, x=17)
>>
>> water_vapor_partial_pressure_in_air(time=19723, y=28, x=17)
>>
>> wind_speed(time=19723, y=28, x=17)
>>
>>
>>
>> Cache Variables
>>
>> time (ucar.nc2.ncml.AggregationOuterDimension$CoordValueVar)
>>
>>
>>
>> Variable Proxies
>>
>> lat cached
>>
>> lon cached
>>
>> crs cached
>>
>> rainfall_amount proxy ucar.nc2.ncml.AggregationExisting
>>
>> duration_of_sunshine proxy ucar.nc2.ncml.AggregationExisting
>>
>> air_temperature proxy ucar.nc2.ncml.AggregationExisting
>>
>> water_vapor_partial_pressure_in_air proxy
>> ucar.nc2.ncml.AggregationExisting
>>
>> wind_speed proxy ucar.nc2.ncml.AggregationExisting
>>
>> x proxy ucar.nc2.dataset.CoordinateAxis1D
>>
>> y proxy ucar.nc2.dataset.CoordinateAxis1D
>>
>> time proxy ucar.nc2.dataset.CoordinateAxis1D
>>
>>
>>
>> So is there an issue with trying to check nested aggregation NcML files
>> using version 4.5 of the netCDF ToolsUI?
>>
>>
>>
>>
>>
>>
>>
>> Furthermore, when I copy the netCDF files and NcML file to a THREDDS
>> 4.5.2 server, restart it, and check the nested aggregation using the
>> OPeNDAP interface the time dimension is displayed as having 38716 values,
>> double the number of time values I’m expecting. When I download the time
>> values using the OPeNDAP interface the time values returned are 0.0, 1.0,
>> 2.0, …, 19355.0, 19356.0, 19357.0, 0.0, 1.0, 2.0, …, 19355.0, 19356.0,
>> 19357.0.
>>
>>
>>
>> Similarly, when I use the NetCDFSubset interface I can download netCDF
>> files for dates between 01/01/1961 to 31/12/2013 but when I try to download
>> a netCDF file for any dates between 01/01/2014 to 31/12/2014 I get a
>> “Requested time 2014-01-01T00:00:00Z does not intersect actual time range
>> 1961-01-01T00:00:00Z – 2013-12-31T00:00:00Z” error.
>>
>>
>>
>> Also, when I use the Godiva2 viewer to view some of the data the date
>> range on the calendar widget allows values between 01/01/1961 to 31/12/2013
>> to be visualised but not any dates between 01/01/2014 to 31/12/2014.
>>
>>
>>
>> Therefore does anyone know if there are issues using a NcML Nested
>> Aggregation with THREDDS 4.5.2? Or am I’m specifying the NcML Nested
>> Aggregation incorrectly?
>>
>>
>>
>>
>>
>>
>>
>> Many thanks for any help that anyone can provide.
>>
>>
>>
>> Best wishes, Simon.
>>
>>
>>
>>
>>
>>
>>
>> Simon Wright
>>
>> Centre for Ecology & Hydrology Lancaster
>> ------------------------------
>> This message (and any attachments) is for the recipient only. NERC is
>> subject to the Freedom of Information Act 2000 and the contents of this
>> email and any reply you make may be disclosed by NERC unless it is exempt
>> from release under the Act. Any material supplied to NERC may be stored in
>> an electronic records management system.
>> ------------------------------
>>
>> _______________________________________________
>> thredds mailing list
>> thredds@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
>
>
> --
> Ryan May
> Software Engineer
> UCAR/Unidata
> Boulder, CO
>
--
Ryan May
Software Engineer
UCAR/Unidata
Boulder, CO
