[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [thredds] announce stable release 4.3.19



Hi, John!

  Thanks for the tip of putting NcML inside the catalog.xml! It looks more 
handy than writing hundreds of individual NcML file and put them under 
dataRoot. 

  I'm just wondering if there's a solution to automate this dynamically without 
writing an explicit NcML style catalog generation  tool  for individual data 
under Hyrax.  I'm talking about a new THREDDS capability that can crawl all  
data available served by Hyrax server(s) and make them ready (as if they are 
local file on THREDDS sever) to serve all  fancy services (WMS, ncISO, netCDF 
subset) that are only available through THREDDS. 

 For example,  I wish THREDDS can handle this kind rule inside catalog.xml:

 <datasetScan name="Remote Hyrax server", ID="testHyraxDataScan"  path="hyrax" 
location="http://hyrax.top.level/opendap/data/";>
    <filter><include wildcard="*.hdf"/></filter> <!-- serve only remote HDF4 
data, not HDF5 -->
</datasetScan>

 Do you think it's a good idea to pursue?

 Regards, 
 

-----Original Message-----
From: John Caron [mailto:address@hidden] 
Sent: Friday, October 18, 2013 11:46 AM
To: Joe Lee
Cc: Ethan Davis; Ted Habermann; address@hidden
Subject: Re: [thredds] announce stable release 4.3.19

Hi Joe:

On our Test server, I have:

<?xml version="1.0" encoding="UTF-8"?>
<catalog name="TDS Test Server"
         xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
         xmlns:xlink="http://www.w3.org/1999/xlink";>

  <service name="all" base="" serviceType="compound">
    <service name="odap" serviceType="OpenDAP" base="/thredds/dodsC/"/>
    <service name="wcs" serviceType="WCS" base="/thredds/wcs/"/>
    <service name="wms" serviceType="WMS" base="/thredds/wms/"/>
    <service name="ncss" serviceType="NetcdfSubset" base="/thredds/ncss/"/>
    <service name="cdmremote" serviceType="CdmRemote" 
base="/thredds/cdmremote/"/>
    <service name="iso" serviceType="ISO" base="/thredds/iso/"/>
    <service name="ncml" serviceType="NCML" base="/thredds/ncml/"/>
    <service name="uddc" serviceType="UDDC" base="/thredds/uddc/"/>   
  </service>


  <dataset name="Test Hyrax" ID="Hyrax2TDS" serviceName="all" 
urlPath="Hyrax2TDS/AIRS.2002.08.01.L3.RetStd_H031.v4.0.21.0.G06104133732.hdf" 
dataType="Grid">
    <ncml:netcdf 
xmlns:ncml="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; 
          
location="http://eosdap.hdfgroup.uiuc.edu:8080/opendap/data/NASAFILES/hdf4/AIRS.2002.08.01.L3.RetStd_H031.v4.0.21.0.G06104133732.hdf";
 />
  </dataset>

</catalog>

which exposes the Hyrax dataset through all the TDS services.
The urlPath is arbitrary but it seems like just echoing the hyrax file name is 
a good thing to do.
you probably would want to generate these catalogs with a script. also may have 
trouble scaling up if you had thousands of them, since the ncml gets cached in 
memory.

Seems to work ok. It would be good to test this thoroughly. Do you have a TDS 
running?

John



On 10/17/2013 1:49 PM, Joe Lee wrote:
> Thanks, John!
> FYI, I succeeded in building TDS-4.3.19 with maven.
> 
>    By the way, I have one question.
> 
>    Do you think it's possible to serve remote data served by Hyrax OPeNDAP 
> server through THREDDS?
> 
>    I know I can write a simple NcML file as a wrapper and put it under 
> THREDDS public/ data directory but I'd like to serve OPeNDAP data directly 
> via URL.
> Since netCDF-Java can access remote OPeNDAP resource via nc_open() call 
> underneath, I think I can simply replace the local file name with URL in 
> theory.
> I'm wondering if anyone has done a similar work of accessing OPeNDAP directly 
> with THREDDS.
> 
>    The purpose is to allow access NASA data served by Hyrax via THREDDS so 
> that users can utilize the THREDDS'  WMS/ncISO/NetCDF subset  capability. So 
> far, many NASA HDF data products  can be visualized by netCDF-Java client 
> like IDV/Panoply already because they follow the CF-conventions through our 
> Hyrax HDF handler work.  My next goal is to integrate THREDDS with Hyrax to 
> provide additional services.
> 
>    Any input or suggestion is welcome!
> 
> -----Original Message-----
> From: John Caron [mailto:address@hidden]
> Sent: Thursday, October 17, 2013 2:24 PM
> To: Joe Lee
> Cc: Ethan Davis; Ted Habermann
> Subject: Re: [thredds] announce stable release 4.3.19
> 
> the gradle stuff is not ready to use yet - expect it will require IntelliJ 
> 13. WE are still using maven, so you should "import maven project".
> 
> On 10/17/2013 1:11 PM, H. Joe Lee wrote:
>> Hi, John!
>>
>>     I tried to build the new Thredds but have an issue with 
>> IntelliJ/Gradle on my Windows 7.
>>
>>     Would you please take a look at my screenshot and tell me what could be 
>> wrong?
>>
>>     I git pulled the source from the documentation link that you sent.
>>
>> --
>> HDF: Software that Powers Science
>>
>>
>> On Tue, Oct 8, 2013 at 5:26 PM, John Caron <address@hidden> wrote:
>>> The Netcdf-Java library and TDS stable release 4.3.19 is now ready:
>>>
>>>     
>>> http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/documentation.htm
>>>     
>>> http://www.unidata.ucar.edu/software/thredds/current/tds/TDS.html
>>>
>>> (note that the above URLs are now the standard way to get the stable 
>>> releases. Please update your bookmarks)
>>>
>>> This is entirely a (minor) bug fix.
>>>
>>>
>>> CDM
>>>    * error assigning gregorian calendar as default to CF files
>>>    * grib: protect against null vertical level name
>>>    * grib time partition : ignore groups not in canonical partition
>>>    * grib: fnmoc tables, try to improve
>>>    * hdf5: deep error in FractalHeap
>>>    * ncml: was ignoring separators in <values> tag
>>>
>>> TDS
>>>    * ncss fixes
>>>    * updated ncWms version (20130918)
>>>
>>>
>>> thanks!
>>>
>>> _______________________________________________
>>> thredds mailing list
>>> address@hidden
>>> For list information or to unsubscribe,  visit:
>>> http://www.unidata.ucar.edu/mailing_lists/