[netcdf-java] NetCDF GridSubsetService Out of memory errors

 
Hi,

I'm getting out of memory errors trying to aggregate some rainfall data
files (each 2.5Mb) using the netcdf grid subsetting service in thredds
(3.16.19) with the following..

<dataset name="Rainfall" ID="RainAggregated"
urlPath="content/bilo/aggregatedData.nc">
 <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
        <aggregation dimName="time" type="joinNew">
          <variableAgg name="rain"/>
          <scan location="c:\data\bilo\2007\" suffix=".nc"
dateFormatMark="#yyyyMMdd" />
        </aggregation>
      </netcdf>

Generally the aggregation works for producing outputs of up to about
50Mb, whereby anything greater generates the following stack trace on
multiple pc's...

Oct 29, 2007 3:34:48 PM org.apache.catalina.core.StandardWrapperValve
invoke
SEVERE: Servlet.service() for servlet GridSubsetService threw exception
java.lang.OutOfMemoryError: Java heap space
 at ucar.ma2.ArrayFloat.<init>(ArrayFloat.java:86)
 at ucar.ma2.ArrayFloat$D3.<init>(ArrayFloat.java:291)
 at ucar.ma2.ArrayFloat$D3.<init>(ArrayFloat.java:283)
 at ucar.ma2.ArrayFloat.factory(ArrayFloat.java:50)
 at ucar.ma2.ArrayFloat.factory(ArrayFloat.java:36)
 at ucar.ma2.Array.factory(Array.java:130)
 at ucar.ma2.Array.factory(Array.java:79)
 at ucar.nc2.ncml.Aggregation.read(Aggregation.java:666)
 at ucar.nc2.dataset.VariableDS._read(VariableDS.java:277)
 at ucar.nc2.Variable.read(Variable.java:603)
 at ucar.nc2.Variable.read(Variable.java:556)
 at ucar.nc2.FileWriter.copySome(FileWriter.java:287)
 at ucar.nc2.FileWriter.copyVarData(FileWriter.java:225)
 at ucar.nc2.FileWriter.finish(FileWriter.java:459)
 at ucar.nc2.dt.grid.NetcdfCFWriter.makeFile(NetcdfCFWriter.java:198)
 at thredds.server.ncSubset.GridServlet.sendFile(GridServlet.java:395)
 at
thredds.server.ncSubset.GridServlet.processGrid(GridServlet.java:354)
 at thredds.server.ncSubset.GridServlet.doGet(GridServlet.java:148)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
 at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
tionFilterChain.java:290)
 at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt
erChain.java:206)
 at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValv
e.java:233)
 at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValv
e.java:175)
 at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(Authenticator
Base.java:433)
 at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java
:128)
 at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java
:102)
 at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.
java:109)
 at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:2
63)
 at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:84
4)
 at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(
Http11Protocol.java:584)
 at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
        
Any suggestions on how to resolve or further diagnose?

Thanks for help,

Trevor



  • 2007 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: