Re: [thredds] THREDDS 4.5.2, ToolsUI 4.5 and nested aggregation issues

  • To: "Wright, Simon M." <smw@xxxxxxxxx>
  • Subject: Re: [thredds] THREDDS 4.5.2, ToolsUI 4.5 and nested aggregation issues
  • From: Ryan May <rmay@xxxxxxxx>
  • Date: Mon, 9 Mar 2015 13:44:30 -0600
Simon,

Thanks for the data. It helped reveal a problem with caching and
aggregations. This should be fixed in the new 4.5.5 release.

Please give a shot and let us know if you're still having problems.

Thanks,

Ryan

On Wed, Mar 4, 2015 at 11:12 AM, Ryan May <rmay@xxxxxxxx> wrote:

> Simon,
>
> That looks like it should work. Can you send me or point me to the actual
> data files so I can try to reproduce with your NCML?
>
> Ryan
>
> On Fri, Feb 27, 2015 at 9:12 AM, Wright, Simon M. <smw@xxxxxxxxx> wrote:
>
>>  Dear All
>>
>>
>>
>> I’m trying to set up a nested aggregation to use in THREDDS 4.5.2 but am
>> having some issues, so wonder if anyone can help.
>>
>>
>>
>> I’m following the Nested Aggregation example at
>> http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/ncml/Aggregation.html
>>
>> and have created the following ncml file:
>>
>>
>>
>> <?xml version='1.0' encoding='UTF-8'?>
>>
>> <netcdf xmlns='http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2'>
>>
>>        <aggregation dimName="time" type="joinExisting">
>>
>>               <netcdf>
>>
>>                      <aggregation type="union">
>>
>>                            <netcdf
>> location="../detail/MORECSDailyRf19610101to20131231.nc" />
>>
>>                            <netcdf
>> location="../detail/MORECSDailySun19610101to20131231.nc" />
>>
>>                            <netcdf
>> location="../detail/MORECSDailyTemp19610101to20131231.nc" />
>>
>>                            <netcdf
>> location="../detail/MORECSDailyVapP19610101to20131231.nc" />
>>
>>                            <netcdf
>> location="../detail/MORECSDailyWSp19610101to20131231.nc" />
>>
>>                      </aggregation>
>>
>>               </netcdf>
>>
>>               <netcdf>
>>
>>                      <aggregation type="union">
>>
>>                            <netcdf
>> location="../detail/MORECSDailyRf20140101to20141231.nc" />
>>
>>                            <netcdf
>> location="../detail/MORECSDailySun20140101to20141231.nc" />
>>
>>                            <netcdf
>> location="../detail/MORECSDailyTemp20140101to20141231.nc" />
>>
>>                            <netcdf
>> location="../detail/MORECSDailyVapP20140101to20141231.nc" />
>>
>>                            <netcdf
>> location="../detail/MORECSDailyWSp20140101to20141231.nc" />
>>
>>                      </aggregation>
>>
>>               </netcdf>
>>
>>        </aggregation>
>>
>> </netcdf>
>>
>>
>>
>> The data in the netCDF files is daily data with time values being
>> relative to 01/01/1961.  For the files suffixed with
>> 19610101to20131231.nc there are 19358 values on the time dimension with
>> values ranging between 0.0 and 19357.0.  For those files suffixed
>> 20140101to20141231.nc there are 365 values on the time dimension with
>> values ranging between 19358.0 and 19722.0.  All other dimensions in the
>> netCDF files – eastings and northings on the British National Grid- are the
>> same.
>>
>>
>>
>> When I check the NcML file using the ToolsUI version 4.5 using NcML and
>> Aggregation tabs I get the following error:
>>
>>
>>
>> java.nio.file.InvalidPathException: Illegal char <:> at index 4
>>
>>
>>
>> However, when I check the ncml file using the ToolsUI version 4.3 again
>> using the NcML and Aggregation tabs I see the aggregation that I expect
>> with 5 union-ed variables with a time dimension having 19358 + 365 = 19723
>> values.  The following summary is displayed in the ToolsUI 4.3 interface:
>>
>>
>>
>>   Type=joinExisting
>>
>>   dimName=time
>>
>>   Datasets
>>
>>
>> file:F:/THREDDS/MORECS/20150227/aggregation/DetailWholeDataset.ncml#4138984
>> range=[0:19358) (19358)
>>
>>
>> file:F:/THREDDS/MORECS/20150227/aggregation/DetailWholeDataset.ncml#25192620
>> range=[19358:19723) (365)
>>
>>   timeUnitsChange=false
>>
>>   totalCoords=19723
>>
>>
>>
>> Aggregation Variables
>>
>>    time(time=19723)
>>
>>    rainfall_amount(time=19723, y=28, x=17)
>>
>>    duration_of_sunshine(time=19723, y=28, x=17)
>>
>>    air_temperature(time=19723, y=28, x=17)
>>
>>    water_vapor_partial_pressure_in_air(time=19723, y=28, x=17)
>>
>>    wind_speed(time=19723, y=28, x=17)
>>
>>
>>
>> Cache Variables
>>
>>    time (ucar.nc2.ncml.AggregationOuterDimension$CoordValueVar)
>>
>>
>>
>> Variable Proxies
>>
>>                     lat cached
>>
>>                     lon cached
>>
>>                     crs cached
>>
>>         rainfall_amount proxy ucar.nc2.ncml.AggregationExisting
>>
>>    duration_of_sunshine proxy ucar.nc2.ncml.AggregationExisting
>>
>>         air_temperature proxy ucar.nc2.ncml.AggregationExisting
>>
>>    water_vapor_partial_pressure_in_air proxy
>> ucar.nc2.ncml.AggregationExisting
>>
>>              wind_speed proxy ucar.nc2.ncml.AggregationExisting
>>
>>                       x proxy ucar.nc2.dataset.CoordinateAxis1D
>>
>>                       y proxy ucar.nc2.dataset.CoordinateAxis1D
>>
>>                    time proxy ucar.nc2.dataset.CoordinateAxis1D
>>
>>
>>
>> So is there an issue with trying to check nested aggregation NcML files
>> using version 4.5 of the netCDF ToolsUI?
>>
>>
>>
>>
>>
>>
>>
>> Furthermore, when I copy the netCDF files and NcML file to a THREDDS
>> 4.5.2 server, restart it, and check the nested aggregation using the
>> OPeNDAP interface the time dimension is displayed as having 38716 values,
>> double the number of time values I’m expecting.  When I download the time
>> values using the OPeNDAP interface the time values returned are 0.0, 1.0,
>> 2.0, …, 19355.0, 19356.0, 19357.0, 0.0, 1.0, 2.0, …, 19355.0, 19356.0,
>> 19357.0.
>>
>>
>>
>> Similarly, when I use the NetCDFSubset interface I can download netCDF
>> files for dates between 01/01/1961 to 31/12/2013 but when I try to download
>> a netCDF file for any dates between 01/01/2014 to 31/12/2014 I get a
>> “Requested time 2014-01-01T00:00:00Z does not intersect actual time range
>> 1961-01-01T00:00:00Z – 2013-12-31T00:00:00Z” error.
>>
>>
>>
>> Also, when I use the Godiva2 viewer to view some of the data the date
>> range on the calendar widget allows values between 01/01/1961 to 31/12/2013
>> to be visualised but not any dates between 01/01/2014 to 31/12/2014.
>>
>>
>>
>> Therefore does anyone know if there are issues using a NcML Nested
>> Aggregation with THREDDS 4.5.2?  Or am I’m specifying the NcML Nested
>> Aggregation incorrectly?
>>
>>
>>
>>
>>
>>
>>
>> Many thanks for any help that anyone can provide.
>>
>>
>>
>> Best wishes, Simon.
>>
>>
>>
>>
>>
>>
>>
>> Simon Wright
>>
>> Centre for Ecology & Hydrology Lancaster
>>  ------------------------------
>> This message (and any attachments) is for the recipient only. NERC is
>> subject to the Freedom of Information Act 2000 and the contents of this
>> email and any reply you make may be disclosed by NERC unless it is exempt
>> from release under the Act. Any material supplied to NERC may be stored in
>> an electronic records management system.
>> ------------------------------
>>
>> _______________________________________________
>> thredds mailing list
>> thredds@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
>
>
> --
> Ryan May
> Software Engineer
> UCAR/Unidata
> Boulder, CO
>



-- 
Ryan May
Software Engineer
UCAR/Unidata
Boulder, CO
  • 2015 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: