Hi all,
Yeap - I'll schedule an outage and upgrade all of our servers.
Thanks for the help!
Cheers,
-Pauline.
Ethan Davis wrote:
Hi Pauline,
Rich wrote:
This was introduced in mid-September, so you may need to upgrade your
TDS.
The fix Rich mentions is only in our 4.1 (development) release, not in
our 4.0 (stable) release. The caching system has undergone some big
changes in 4.1 so I'm not sure if this fix will ever make it into 4.0.
Can you upgrade this system to TDS 4.1?
Ethan
Richard Signell wrote:
Pauline,
To keep files that are overwritten with new data from being cached in
the TDS, you can add a tag cache="false" in the datasetRoot element.
This was introduced in mid-September, so you may need to upgrade your
TDS. Here's an example catalog which serves two ocean model forecast
files. These forecast files are replaced every day with new files,
but they always have the same name.
<?xml version="1.0" encoding="UTF-8"?>
<catalog xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
xmlns:xlink="http://www.w3.org/1999/xlink"; name="THREDDS Catalog for
NetCDF Files" version="1.0.1">
<service name="allServices" serviceType="Compound" base="">
<service name="ncdods" serviceType="OpenDAP" base="/thredds/dodsC/"/>
<service name="HTTPServer" serviceType="HTTPServer"
base="/thredds/fileServer/"/>
</service>
<datasetRoot path="FVCOM" location="/http/www/CODFISH/Data/FVCOM/"
cache="false"/>
<dataset name="GOM2 Forecast with no caching" ID="gom2_nocache"
serviceName="allServices"
urlPath="FVCOM/NECOFS/Forecasts/NECOFS_GOM2_FORECAST.nc" dataType="Grid"/>
<dataset name="MASSBAY Forecast with no caching"
ID="massbay_nocache" serviceName="allServices"
urlPath="FVCOM/NECOFS/Forecasts/NECOFS_FVCOM_OCEAN_MASSBAY_FORECAST.nc"
dataType="Grid"/>
</catalog>
-Rich
On Wed, Dec 2, 2009 at 12:20 AM, Pauline Mak <pauline.mak@xxxxxxxxxxx> wrote:
Hi all,
I have a user of TDS who had updated some files on the server and it seems
like TDS is caching this and not refreshing it. Has anyone else come across
this problem?
The directory we're looking at is:
http://opendap-ivec.arcs.org.au/thredds/catalog/IMOS/SRS/SeaDAS/test/
As a test we copied one of the problematic files
(A2009100.0320.L2_OCEANGRID.hdf) and called it test.hdf, and it works fine!
Note that, while the files are identical, the metadata in the global
attributes is different:
new file: time: "2009-04-10T03:22:38Z"
old file: time: "2009-03-10T03:22:38Z"
Interestingly, for A2009100.0320.L2_OCEANGRID.hdf, it seems like there's
problem with the dimensions (which would explain why WCS and WMS doesn't
work). When I tried to retrieve the first 2000 elements in latitude, it
worked fine (I got the ascii response). However, getting 2200, out of a
possible 2437, I get the stacktrace error attached.
I am using version 4.0.26 - 20090831.2140 running on Jetty 6.1.18, with the
following cache settings in TDS:
<DiskCache>
<alwaysUse>true</alwaysUse>
<dir>/data/tmp/thredds/cache/</dir>
<scour>1 hour</scour>
<maxSize>10 Gb</maxSize>
</DiskCache>
<NetcdfFileCache>
<minFiles>200</minFiles>
<maxFiles>400</maxFiles>
<scour>10 min</scour>
</NetcdfFileCache>
<NetcdfDatasetCache>
<minFiles>100</minFiles>
<maxFiles>200</maxFiles>
<scour>10 min</scour>
</NetcdfDatasetCache>
<NetcdfSubsetService>
<allow>true</allow>
<dir>/data/tmp/thredds/ncSubsetCache/</dir>
<scour>10 min</scour>
<maxAge>-1 min</maxAge>
<metarDataDir>/opt/tomcat/content/thredds/public/stn/</metarDataDir>
<metarRawDir>/data/ldm/pub/decoded/netcdf/surface/metar/</metarRawDir>
</NetcdfSubsetService>
<WCS>
<allow>true</allow>
<allowRemote>false</allowRemote>
<dir>/tmp/thredds/wcsCache/</dir>
<scour>15 min</scour>
<maxAge>30 min</maxAge>
</WCS>
<WMS>
<allow>true</allow>
<paletteLocationDir>/opt/tds/webapps/thredds/WEB-INF/palettes</paletteLocationDir>
</WMS>
Thanks!
-Pauline.
--
Pauline Mak
Assistant Manager, ARCS Data Services
Ph: +61 3 6226 7518
Mob: +61 411 638 196
Email: pauline.mak@xxxxxxxxxxx
Jabber: pauline.mak@xxxxxxxxxxx
Calendar: http://tinyurl.com/pmak-arcs-calendar
http://www.arcs.org.au/
TPAC
Email: pauline.mak@xxxxxxxxxxx
http://www.tpac.org.au/
java.lang.RuntimeException: NcSDArray
ArrayIndexOutOfBoundsException=NcSDArray read Latitude Latitude(0,1,2200)
null dataset= IMOS/SRS/SeaDAS/test/A2009100.0320.L2_OCEANGRID.hdf
at thredds.server.opendap.NcSDArray.read(NcSDArray.java:118)
at opendap.servlet.AsciiWriter.writeAsc(AsciiWriter.java:95)
at opendap.servlet.AsciiWriter.toASCII(AsciiWriter.java:56)
at
thredds.server.opendap.OpendapServlet.doGetASC(OpendapServlet.java:320)
at
thredds.server.opendap.OpendapServlet.doGet(OpendapServlet.java:222)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at thredds.servlet.filter.CookieFilter.doFilter(CookieFilter.java:54)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1148)
at
thredds.servlet.filter.RequestQueryFilter.doFilter(RequestQueryFilter.java:121)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1148)
at
thredds.servlet.filter.RequestPathFilter.doFilter(RequestPathFilter.java:105)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1148)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:387)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:539)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)
_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
--
Pauline Mak
Assistant Manager, ARCS Data Services
Ph: +61 3 6226 7518
Mob: +61 411 638 196
Email: pauline.mak@xxxxxxxxxxx
Jabber: pauline.mak@xxxxxxxxxxx
Calendar: http://tinyurl.com/pmak-arcs-calendar
http://www.arcs.org.au/
TPAC
Email: pauline.mak@xxxxxxxxxxx
http://www.tpac.org.au/
