Hi Heiko:
Can you see whether the second time you access the dataset, if the times
are fast again?
thanks,
John
On Wed, May 20, 2015 at 2:07 AM, Heiko Klein <Heiko.Klein@xxxxxx> wrote:
> Hi,
>
> I have some performance problems after upgrading to thredds 4.6.1.
>
> I'm aggregating a large dataset with a joinExisting aggregation. Reading
> the metadata from the aggregation took, with thredds 4.3.23, about 0.1s
> (first time up to 7s). After upgrading to 4.6.1, the same request takes 20s
> (second time, first time not measured) and is unusable slow. A 'ncview' of
> the aggregated dataset is no longer possible.
>
> Suspecting some caching problems, I followed the guidelines in
> http://www.unidata.ucar.edu/software/thredds/current/tds/reference/ThreddsConfigXMLFile.html
> The aggregation-cache contains two files
>
> $ ls -l file-lustre-mnt-heikok-FAUNA-mbr000_all.ncml
> lustre/mnt/heikok/FAUNA/mbr000_all.ncml
> -rw-r--r-- 1 tomcat7 tomcat7 74339 May 20 09:43
> file-lustre-mnt-heikok-FAUNA-mbr000_all.ncml
> -rw-r--r-- 1 tomcat7 tomcat7 74131 May 20 10:01
> lustre/mnt/heikok/FAUNA/mbr000_all.ncml
>
> (I guess the first one is thredds 4.3, while the second one is thredds 4.6)
>
>
> The ncml-file is:
>
> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
> <aggregation dimName="time" type="joinExisting">
> <scan location="." regExp=".*snapMet000\.nc$" subdirs="true"/>
> </aggregation>
> </netcdf>
>
>
> It is aggregating 4575 files, each about 1.3G.
>
>
> The requests according to the logs are:
> 2015-05-20T09:49:17.111 +0200 [ 372240][ 281] INFO -
> threddsServlet - Remote host: 157.249.113.42 - Request: "GET
> /thredds/dodsC/lustreMnt/heikok/FAUNA/mbr000_all.ncml.dds HTTP/1.1"
> 2015-05-20T09:49:17.116 +0200 [ 372245][ 281] INFO -
> threddsServlet - Request Completed - 200 - -1 - 5
> 2015-05-20T09:49:17.117 +0200 [ 372246][ 282] INFO -
> threddsServlet - Remote host: 157.249.113.42 - Request: "GET
> /thredds/dodsC/lustreMnt/heikok/FAUNA/mbr000_all.ncml.das HTTP/1.1"
> 2015-05-20T09:49:17.131 +0200 [ 372260][ 282] INFO -
> threddsServlet - Request Completed - 200 - -1 - 14
> 2015-05-20T09:49:17.134 +0200 [ 372263][ 283] INFO -
> threddsServlet - Remote host: 157.249.113.42 - Request: "GET
> /thredds/dodsC/lustreMnt/heikok/FAUNA/mbr000_all.ncml.dds HTTP/1.1"
> 2015-05-20T09:49:17.138 +0200 [ 372267][ 283] INFO -
> threddsServlet - Request Completed - 200 - -1 - 4
>
>
> Best regards,
>
> Heiko
>
>
> --
> Dr. Heiko Klein Norwegian Meteorological Institute
> Tel. + 47 22 96 32 58 P.O. Box 43 Blindern
> http://www.met.no 0313 Oslo NORWAY
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
