Re: [thredds] remote NcML "union" aggregation broken in TDS 4.3?

One follow-up to this. Rich Signell discovered a workaround: if my TDS
catalog references a separate NcML file that contains the NcML aggregation,
it works. However, attempting to do the aggregation within the TDS catalog
still fails (seems to be a bug). Below are examples of what works (#1) and
what doesn't (#2). This is using TDS 4.3.16 - 20130319.1353.

1. This works; put the aggregation into a separate .ncml file (test.ncml):

<?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
  <attribute name="title" value="HYCOM test: temp and salinity union"/>
  <aggregation type="union">
    <netcdf location="dods://
apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomT
"/>
    <netcdf location="dods://
apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomS
"/>
  </aggregation>
</netcdf>

Then reference this .ncml file inside the TDS catalog:

<?xml version="1.0" encoding="UTF-8" ?>

<catalog name="HYCOM Global Ocean Model"
         xmlns="
http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
         xmlns:xlink="http://www.w3.org/1999/xlink"; >

  <service name="gridServices" serviceType="Compound" base="">
    <service name="opendap" serviceType="OpenDAP"
base="/thredds/dodsC/"     />
    <service name="ncss"    serviceType="NetcdfSubset"
base="/thredds/ncss/grid/" />
    <service name="wms"     serviceType="WMS"
base="/thredds/wms/"       />
    <service name="wcs"     serviceType="WCS"
base="/thredds/wcs/"       />
    <service name="iso"     serviceType="ISO"
base="/thredds/iso/"       />
    <service name="ncml"    serviceType="NCML"
base="/thredds/ncml/"      />
  </service>

  <dataset name="HYCOM Global Ocean Model"
           ID="hycom_global"
           urlPath="hycom_global">

    <metadata inherited="true">
      <serviceName>gridServices</serviceName>
      <dataType>Grid</dataType>
    </metadata>

    <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";
location="file:/usr/local/tomcat-tdsdev/content/thredds/idd/test.ncml"/>

  </dataset>

</catalog>

2. This doesn't work: trying to do the NcML aggregation inside the TDS
catalog:

<?xml version="1.0" encoding="UTF-8" ?>

<catalog name="HYCOM Global Ocean Model"
         xmlns="
http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
         xmlns:xlink="http://www.w3.org/1999/xlink"; >

  <service name="gridServices" serviceType="Compound" base="">
    <service name="opendap" serviceType="OpenDAP"
base="/thredds/dodsC/"     />
    <service name="ncss"    serviceType="NetcdfSubset"
base="/thredds/ncss/grid/" />
    <service name="wms"     serviceType="WMS"
base="/thredds/wms/"       />
    <service name="wcs"     serviceType="WCS"
base="/thredds/wcs/"       />
    <service name="iso"     serviceType="ISO"
base="/thredds/iso/"       />
    <service name="ncml"    serviceType="NCML"
base="/thredds/ncml/"      />
  </service>

  <dataset name="HYCOM Global Ocean Model"
           ID="hycom_global"
           urlPath="hycom_global">

    <metadata inherited="true">
      <serviceName>gridServices</serviceName>
      <dataType>Grid</dataType>
    </metadata>

    <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
      <attribute name="title" value="HYCOM test: temp and salinity union"/>
      <aggregation type="union">
        <netcdf location="dods://
apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomT
"/>
        <netcdf location="dods://
apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomS
"/>
      </aggregation>
    </netcdf>

  </dataset>

</catalog>



On Fri, Apr 19, 2013 at 10:35 AM, John Maurer <jmaurer@xxxxxxxxxx> wrote:

> Hi all,
> I'm attempting to do an NcML "union" aggregation of several variables from
> a forecast model that are contained in separate remote OPeNDAP locations.
> While this works in TDS 4.2 (and Tools UI 4.2), I'm getting an error in TDS
> 4.3.16 (and Tools UI 4.3). Below are the details. Perhaps this is related
> to the recent "joinExisting" remote aggregation problem that Roland
> Schweitzer reported in TDS 4.3 here:
> https://www.unidata.ucar.edu/mailing_lists/archives/thredds/2013/msg00093.html.
> I have already tried clearing the cache and restarting Tomcat as was
> suggested in that chain: also did not solve the problem in my case, either.
> Thanks in advance for your help on this.
> Cheers,
> John Maurer
> Pacific Islands Ocean Observing System (PacIOOS)
> University of Hawaii at Manoa
>
> TDS catalog:
>
> <?xml version="1.0" encoding="UTF-8" ?>
>
> <catalog name="HYCOM Global Ocean Model"
>          xmlns="
> http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
>          xmlns:xlink="http://www.w3.org/1999/xlink"; >
>
>   <service name="gridServices" serviceType="Compound" base="">
>     <service name="opendap" serviceType="OpenDAP"
> base="/thredds/dodsC/"     />
>     <service name="ncss"    serviceType="NetcdfSubset"
> base="/thredds/ncss/grid/" />
>     <service name="wms"     serviceType="WMS"
> base="/thredds/wms/"       />
>     <service name="wcs"     serviceType="WCS"
> base="/thredds/wcs/"       />
>     <service name="iso"     serviceType="ISO"
> base="/thredds/iso/"       />
>     <service name="ncml"    serviceType="NCML"
> base="/thredds/ncml/"      />
>   </service>
>
>   <dataset name="HYCOM Global Ocean Model"
>            ID="hycom_global"
>            urlPath="hycom_global">
>
>     <metadata inherited="true">
>       <serviceName>gridServices</
> serviceName>
>       <dataType>Grid</dataType>
>     </metadata>
>
>     <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2
> ">
>       <attribute name="title" value="HYCOM test: temp and salinity union"/>
>
>       <aggregation type="union">
>         <netcdf location="dods://
> apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomT
> "/>
>         <netcdf location="dods://
> apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomS
> "/>
>       </aggregation>
>     </netcdf>
>
>   </dataset>
>
> </catalog>
>
>
> threddsServlet.log:
>
> 2013-04-19T09:29:09.165 -1000 [   1119756][       8] INFO  -
> threddsServlet - Remote host: 128.171.151.240 - Request: "GET
> /thredds-test/dodsC/hycom_
> global_temp.html HTTP/1.1"
> 2013-04-19T09:29:09.328 -1000 [   1119919][       8] ERROR -
> thredds.server.opendap.OpendapServlet - path= /hycom_global_temp.html
> java.lang.NullPointerException
>         at ucar.nc2.util.URLnaming.resolveFile(URLnaming.java:238)
>         at ucar.nc2.ncml.NcMLReader.readAgg(NcMLReader.java:1348)
>         at ucar.nc2.ncml.NcMLReader.readNetcdf(NcMLReader.java:491)
>         at ucar.nc2.ncml.NcMLReader.readNcML(NcMLReader.java:438)
>         at ucar.nc2.ncml.NcMLReader.readNcML(NcMLReader.java:351)
>         at
> thredds.servlet.DatasetHandler$NcmlFileFactory.open(DatasetHandler.java:261)
>         at
> thredds.servlet.DatasetHandler$NcmlFileFactory.open(DatasetHandler.java:252)
>         at ucar.nc2.util.cache.FileCache.acquire(FileCache.java:224)
>         at
> ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:685)
>         at
> ucar.nc2.dataset.NetcdfDataset.acquireFile(NetcdfDataset.java:622)
>         at
> thredds.servlet.DatasetHandler.getNetcdfFile(DatasetHandler.java:158)
>         at
> thredds.server.opendap.OpendapServlet.getDataset(OpendapServlet.java:913)
>         at
> thredds.server.opendap.OpendapServlet.doGetHTML(OpendapServlet.java:613)
>         at
> thredds.server.opendap.OpendapServlet.doGet(OpendapServlet.java:230)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>         at
> thredds.servlet.filter.CookieFilter.doFilter(CookieFilter.java:54)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>         at
> thredds.servlet.filter.RequestQueryFilter.doFilter(RequestQueryFilter.java:120)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>         at
> thredds.servlet.filter.RequestPathFilter.doFilter(RequestPathFilter.java:102)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>         at
> thredds.server.RequestBracketingLogMessageFilter.doFilter(RequestBracketingLogMessageFilter.java:48)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>         at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
>         at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
>         at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>         at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
>         at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
>         at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
>         at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>         at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>         at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987)
>         at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)
>         at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>         at java.lang.Thread.run(Unknown Source)
> 2013-04-19T09:29:09.331 -1000 [   1119922][       8] INFO  -
> threddsServlet - Request Completed - 500 - -1 - 166
>
>
  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: