Re: metadata question

Hi Bruce,

Bruce Flynn wrote:

Hi Ethan - I hope your prepared for this!

Specifically, the metadata element that does not work with the TDS HTML is <metadata xlink:href="http://sands01.ssec.wisc.edu:8080/thredds/Shelves/7a75/4fd8/7a754fd85b39c6b6795936bbcd26955d/"; xlink:title="Parent Dataset" />.

It fails with the error output found at http://www.ssec.wisc.edu/~brucef when navigating to the dataset containing the above metadata. The above XML works if you change the tag to documentation and place it in a metadata element. ???
One problem is that your catalog is not valid according to the InvCatalog schema. The TDS is pretty lenient when reading catalogs so I don't think any of this is causing the problem you are seeing. But here are the details. Just a few changes fix it. At the catalog level, service elements must come before dataset elements. The datasetRoot element needs a location attribute instead of a dirLocation attribute (either is allowed in datasetScan but not in datasetRoot). Inside of datasets, access elements come after metadata and dataSize elements. Also, metadata elements have to come before dataset elements. I've attached a new version of your catalog that fixes these problems.

I think the main problem is that the resource you are linking to (looks like your TDS is down currently, so I haven't actually been able to look at it) appears to not be well formed. Here's the part of the error message that leads me to this conclusion:

   **Error: InvCatalogFactory.readOtherXML failed on
   
http://sands01.ssec.wisc.edu:8080/thredds/Shelves/7a75/4fd8/7a754fd85b39c6b6795936bbcd26955d/

   Exception= org.xml.sax.SAXParseException The element type "meta"
   must be terminated by the matching end-tag "</meta>".

   ** failed to read thredds metadata at
   
http://sands01.ssec.wisc.edu:8080/thredds/Shelves/7a75/4fd8/7a754fd85b39c6b6795936bbcd26955d/
   for datasetmichelle.tar


One other thing that might cause problems. It looks like the TDS HTML view only displays a link for the metadata xlink:href if that is the only metadata element. Sounds like a TDS bug, so we'll look into that.

Can you try the attached catalog and check for well-formed XML in the linked resource and see how well that works?

What I am trying to accomplish is kinda hard to explain, but I'll give it a shot...

I have 2 parallel TDS representations for the same data. There is a "Shelves" tree generated using datasetScan that is an exact recreation of our datastore back-end on our XSAN. The data-store is in a format that is not user navigable, so for users to get to their data we provide soft-links to something more "friendly," which we call the "user links." The XSAN back-end will never change, but the user links might. I have python scripts that generate TDS catalogs from the user links that replicate the soft-linked user navigable access to the XSAN back-end.

So... If a link to a particular dataset(ie. netcdf file) is distributed from one user to another, that user may also be interested in the related files in the parent dataset. Therefore, there was a requirement to have a link to the parent dataset containing related files. Since the user links may change it made sense to provide that link to the shelves tree, which is static.

For example, the user links URL for the above metadata tag is http://localhost:8080/thredds/dodsC/User_Links/Users/johnm/catalog.html, which actually represents the same data.

So, it seems unnatural to classify the link to a parent dataset as documentation.
Well, I'm not sure I really understand but it sounds like you want to explicitly capture a parent/child relationship that the THREDDS catalogs being produced don't capture. The only way to capture this information would be to build a single THREDDS catalog with the parent/child relationships showing up as the hierarchical structure of the catalog. The reason we don't have a place for this kind of information outside of the hierarchical structure is because a catalog can be pointed to from multiple locations so there isn't necessarily a single parent to any given catalog.

Since there isn't an explicitly defined way of doing this in THREDDS catalogs and it sounds like you simply want this information displayed to the user rather than making this a standard, machine readable bit of information, I would suggest using the documentation element. (Here's my thinking on this ... Both documentation and metadata elements are simply providing additional information about the dataset. The difference is the intended audience: a human user, in which case the free-text and/or XHTML of the documentation element should be used; or a machine, in which case the structured information of the metadata element should be used.)

You could just stick some text and an HTML link into a documentation element:

<documentation>Other related datasets are available in this datasets <a href="???">parent collection</a>.</documentation>

or as we've been discussing, an xlink to another document.

Hope this all helps.

Ethan

--
Ethan R. Davis                                Telephone: (303) 497-8155
Software Engineer                             Fax:       (303) 497-8690
UCAR Unidata Program Center                   E-mail:    edavis@xxxxxxxx
P.O. Box 3000
Boulder, CO  80307-3000                       http://www.unidata.ucar.edu/
---------------------------------------------------------------------------

<?xml version="1.0" encoding="UTF-8"?>


<catalog xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
         xmlns:xlink="http://www.w3.org/1999/xlink";>

  <service name="group" base="" serviceType="Compound">

    <service name="dods" base="/thredds/dodsC/" serviceType="OpenDAP"/>

    <service name="http" base="/thredds/fileServer/" serviceType="HTTPServer"/>

    <datasetRoot path="Shelves" location="/Volumes/gulf1/.bruce_dev/"/>

  </service>

  <service name="other" base="/thredds/fileServer/" serviceType="HTTPServer"/>

  <dataset name="7a754fd85b39c6b6795936bbcd26955d" 
ID="7a754fd85b39c6b6795936bbcd26955d">

    <metadata inherited="true">

      <documentation type="summary">

        MM5 code for michelle dataset setup by John M

      </documentation>

      <keyword>MM5</keyword>

      <keyword>michelle</keyword>

      <keyword>Jan2003</keyword>

      <property name="link" value="Users/johnm"/>

      <creator>

        <name>eriko</name>

        <contact email="eriko@xxxxxxxxxxxxx"/>

      </creator>

      <publisher>

        <name>Administrator</name>

        <contact email="harvester_admin@xxxxxxxxxxxxx"/>

      </publisher>

    </metadata>

    <dataset name="michelle.tar" 
ID="7a754fd85b39c6b6795936bbcd26955d-michelle.tar">

      <metadata>

        <documentation>

          The dataset in which this file is contained is available at:

        </documentation>

        <property name="MD5 Checksum" value="a41252c9243d6ee8b66ff353736a3fc4"/>

        <dataFormat>text/plain</dataFormat>

      </metadata>

      <metadata
              
xlink:href="http://sands01.ssec.wisc.edu:8080/thredds/Shelves/7a75/4fd8/7a754fd85b39c6b6795936bbcd26955d/";
              xlink:title="Parent Dataset"/>

      <dataSize units="bytes">1984233675</dataSize>

      <access 
urlPath="Shelves/7a75/4fd8/7a754fd85b39c6b6795936bbcd26955d/michelle.tar" 
serviceName="other"/>

    </dataset>

  </dataset>

</catalog>
  • 2006 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: