[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #QIL-297611]: aggregation of remote datasets CrawlableDatasetDods

Hi Tyn,

> > I'm not sure I understand what you mean by "without THREDDS".
> 
> sorry for fuzzy use of wording. i meant to emphasize it's
> ability to catalog an remote OPeNDAP server, without
> 'harvesting' a remote catalog relying on the remote machine
> own catalog THREDDS service. Hence the term, 'without THREDDS'.
> 
> But you understand me.

Yes, I think I understand now.

> > 1) The TDS will crawl OPeNDAP servers and generate catalogs using 
> > CrawlableDatasetDods.
> >
> > 2) CrawlableDatasetDods can be used programatically to crawl OPeNDAP 
> > servers.
> 
> What functionality is added/different in 2 vs. 1? Can you give an example?

My turn to explain my fuzzy comments. The difference I was trying to point out 
wasn't functional but rather usage. Item 1 has to do with using 
CrawlableDatasetDods in TDS with some fairly simple additions to the TDS 
configuration. Item 2 is about writing new Java code to use 
CrawlableDatasetDods.
 
> >> Any ideas how to get this going, perhaps Bas can help again?
> >>
> > Again, I'm not sure I'm understanding. Is something
> > that was working no longer working?
> 
> yes, the aggregation. Any ETA? Can Bas help you with this?

I just made a few changes in our code that should allow aggregation to work 
with CrawlableDatasetDods. However, I don't have any URLs for old OPeNDAP 
servers of the type that CrawlableDatasetDods works with. Do you have any? Are 
you still serving data using CrawlableDatasetDods in datasetScan elements?

> Other than that, I recall Bas struggeling getting your thredds.war
> working as CrawlableDatasetDods class files were zipped. But before
> bothering you with this, and as it is a long time ago, let me first
> try and experiment with the latest TDS 4.0.9. I will report back my
> results. Is there an example catalog.xml which you have been using
> to test CrawlableDatasetDods?

Have you had any time to try things on your end? We now have 4.0.14 out if you 
want to try with that. Though there aren't any changes between 4.0.9 and 4.0.14 
having to do with CrawlableDatsaetDods.

I don't have any test catalogs for testing CrawlableDatasetDods. Since most of 
the old C++ OPeNDAP servers went away after the security notice, I haven't 
found any for testing. Do you guys still point to servers that work with 
CrawlableDatasetDods?

Thanks,

Ethan


Ticket Details
===================
Ticket ID: QIL-297611
Department: Support THREDDS
Priority: Normal
Status: Open