John, Yes, unfortunately I think the community has frequently been seeing these types of problems with large TDS aggregations. The first unlucky user (or client) to access data after the metadata in the AggregationCache has been cleared is going to have a bad experience, waiting a long time (and perhaps timing out) before the metadata cache is rebuilt. I just tried accessing the OSCAR surface current aggregation from the JPL PO.DAAC TDS at: <http://thredds.jpl.nasa.gov/thredds/podaac_catalogs/OSCAR_L4_OC_third-deg_catalog.html?dataset=ALL_OSCAR_L4_OC_third-deg> and here's what happened: - The first time (5:30 EST) I selected the opendap access link, I waited for 5 minutes, and then I got a 503 error (service temporarily unavailable or capacity problem). - The second time (5:35 EST) I selected the opendap access link, after 4.5 minutes, the OPeNDAP Access Form (finally) appeared. - The third time (5:36 EST) I selected the opendap access link, it responded in less than a second. - After that, of course, the WMS also responded very quickly (and reminded me that this is a cool dataset, godiva2 image attached) I think the most likely problem is that the aggregationCache is getting scoured. On the page describing the AggregationCache setting in threddsConfig.xml, <https://www.unidata.ucar.edu/projects/THREDDS/tech/reference/ThreddsConfigXMLFile.html#AggregationCache> it says: "If you have joinExisting Aggregations, coordinate information will be written to a cache directory....Every scour amount of time, any item that hasnt been changed since maxAge time will be deleted. Set scour to -1 to not scour if you have aggregations that never change." The OSCAR dataset is not changing, so JPL could modify their threddsConfig.xml to be: ... <AggregationCache> ... <scour>-1 hours</scour> ... </AggregationCache> I'm cc'ing the THREDDS list as it seems a common issue. -Rich On Thu, Feb 14, 2013 at 4:21 PM, John Maurer <jmaurer@xxxxxxxxxx> wrote: > Hi Rich, > I was wondering if you've had any experience using > JPL/PO.DAAC's THREDDS Data Server?? > > http://thredds.jpl.nasa.gov/thredds/catalog.html > > There are a few things on here that my team is interested in integrating in > my viewer (e.g. G1SST, OSTIA, OSCAR), but everything I try runs painfully > slow or never responds at all (particularly WMS is what I've been playing > with). My guess is that the aggregations are so large it grinds TDS nearly > to a halt. I've tried e-mailing JPL to do some performance tuning, but they > seem to be in denial about the problem and stopped responding. Have you had > any success pulling in data from them?? > Aloha, > John Maurer -- Dr. Richard P. Signell (508) 457-2229 USGS, 384 Woods Hole Rd. Woods Hole, MA 02543-1598
Attachment:
2-17-2013 6-05-05 AM.png
Description: PNG image
thredds archives: