Hi,
I looked a bit further into this. I found that the DODSNetcdfFile upon
construction also started to cache some string and coordinate variables.
That's why the GET
/opendap/rijkswaterstaat/jarkus/transect.nc.dods?areaname,year HTTP/1.1
request is done.
The toolsui library is used in the matlab snctools library. This matlab
library constructs the DODSNetcdfFile object at each variable or metadata
request.
I solved this temporary by making the setPreload method public and using
this from matlab, but as it states in the code this is not intended for use
other then debugging.
There are possible ways to improve performance when using the DODSNetcdfFile
and snctools:
- remove preloading from constructor in DODSNetcdfFile
This will decrease performance on applications which assume the
DODSNetcdfFile to be able to cache.
- option 1 and make preloading a decorator (comparable to bufferedReader in
java.io)
new PreloadingFile(new DODSNetcdfFile(url))
- use a caching or loading strategy, perhaps static:
DODSNetcdfFile.setLoadingStrategy(new PreloadStrategy())
- make preloading optional through an argument
new DODSNetcdfFile(url, null, false)
- make preloading setable
DODSNetcdfFile.setPreloading(false)
- cancel at the preloading step. This can now be done with a canceltask but
I don't quite see how to use this.
- make snctools use connection pooling so it wont open a new DODSNetcdfFile
at each function call.
Personaly I think caching should be optional to allow uses like snctools
uses it now. The first three options may decrease performance for existing
applications but seem to me most elegant. The fourth and fifth option don't
require any change of existing code but this doesn't realy fit a "java
style" of programming. The 6th I do not understand and the 7th leaves it up
to the snctools.
How do you look at this?
Kind regards,
Fedor Baart
On Tue, Nov 11, 2008 at 9:42 PM, fedor baart <f.baart@xxxxxxxxx> wrote:
> Hi,
>
> I am working on a European project Micore for which I setup an opendap
> server. I am experiencing some performance issues.
>
> I tried using the toolsUI-2.2.22.jar with mexnc and matlab to access data
> and noticed that for every request it takes about 10 seconds to call a
> function.
>
> Also when I used the toolsUI as a ui and tried using the ncdump function it
> took about 10 seconds for it to return.
>
> Using wireshark I stored the connection information which contains the
> following information:
>
> after 0.2 seconds:
> GET /opendap/rijkswaterstaat/jarkus/transect.nc.dds HTTP/1.1
> returns after 0.21 seconds with HTTP OK and dimensions
> after 0.22 seconds:
> GET /opendap/rijkswaterstaat/jarkus/transect.nc.das HTTP/1.1
> returns after 0.23 seconds with HTTP OK and attributes
> after 0.24 seconds:
> GET /opendap/rijkswaterstaat/jarkus/transect.nc.dods?areaname,year HTTP/1.1
> after 10 seconds returns with HTTP OK and a lot of data
>
> So what happens:
> 0.25 seconds is spend enquiring the information needed for the ncdump
> output.
> 10 seconds is spend on reading some variables, information which is not
> needed
>
> The site I tried this with is
>
> http://micore.wldelft.nl/opendap/rijkswaterstaat/jarkus/transect.nc.html
>
> I also found the same behaviour though with different variables on another
> site I tried:
>
> http://penguin.sfos.uaf.edu/opendap/FNMOC/sst/nc/200810/sst_2008102406.nc.bz2.html
>
> toolsUI-4.0 gives the same results
>
> Is anyone experiencing similar behaviour?
> Can I assume this to be an error?
> If it is an error in which source code (version 2.2.22 or 4.0 or svn?)
> should it be fixed?
> Can anyone suggest a way to speed up my queries to the opendap server if
> this is expected behaviour?
>
> Thanks,
>
> Fedor Baart
>
