Re: [thredds] Using HDF5's dynamic loading filter in THREDDS

  • To: Christian Ward-Garrison <cwardgar@xxxxxxxx>
  • Subject: Re: [thredds] Using HDF5's dynamic loading filter in THREDDS
  • From: Michael McDonald <mcdonald@xxxxxxxxxxxxx>
  • Date: Thu, 28 Apr 2016 12:33:50 -0400
Adam and Christian,

HYCOM.org just needed to build netcdf+hdf5 from source after
discovering a fatal bug in the latest RHEL6 hdf5 (from epel), which
was crashing our NCSS NetcdfSubsetService servers. The only part we
*added* to the NetCDF-4 C Library Loading reference
(http://bit.ly/1QD34bK) - in addition to "--enable-threadsafe" - was
"--enable-unsupported" in the build of hfd5.

1st remove the rpm build of netcdf+hdf5 from RHEL yum (if that's what
you are using. as we were).

yum remove netcdf netcdf-static netcdf-devel hdf5 hdf5-static hdf5-devel


Then download and build hfd5 1st and then netcdf 2nd from source.

cd /usr/local/src
wget http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.16.tar.gz
tar -zxf hdf5-1.8.16.tar.gz && cd hdf5-1.8.16
./configure --prefix=/usr --enable-threadsafe --enable-unsupported &&
make && make check && make install

ldconfig

cd /usr/local/src
wget https://github.com/Unidata/netcdf-c/archive/v4.4.0.tar.gz
tar -zxf v4.4.0.tar.gz && cd netcdf-c-4.4.0
./configure --prefix=/usr && make && make check && make install


After restarting tomcat the new netcdf bins/libs are working fine to
stage netcdf4 files to cache again for NCSS, and more importantly no
longer crashing tomcat in the process.

On Thu, Apr 28, 2016 at 2:55 AM, Christian Ward-Garrison
<cwardgar@xxxxxxxx> wrote:
> Hi Adam,
>
> THREDDS uses the NetCDF C library to write (but not read) NetCDF-4 files.
> This page [1] describes the procedure that THREDDS uses to locate the shared
> library on the system. Have you successfully built netcdf while linking
> against your custom hdf? Does your hdf require any changes to netcdf in
> order to take advantage of the new features?
>
> Cheers,
> Christian
>
> [1]
> https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/reference/netcdf4Clibrary.html
>
> On Wed, Apr 27, 2016 at 5:51 PM, Adam Steer <Adam.Steer@xxxxxxxxxx> wrote:
>>
>> Hi all
>>
>> I’ve got a request to enable HDF5’s dynamic loading filters capability in
>> a TDS. My colleagues have built a HDF5 C library which has dynamic loading
>> filters enabled, required because the standard HDF5 compression algorithm
>> isn’t working as well as some others they’ve tested and they want to replace
>> it. I can’t answer any more about the whys and wherefores of that - but the
>> task from my point of view is this:
>>
>> How can I get THREDDS to use a customised HDF5 library?
>>
>> As far as I can tell, I’d need to build THREDDS from source, with the new
>> HDF5 library on whatever system I’m using to build it.
>>
>> …but this strategy might be completely wrong. I’m uncertain about the
>> relationship between system NetCDF/HDF5 libraries and the Java-netCDF that
>> THREDDS uses.
>>
>> …which brings my second question: Would using a customised HDF5 library at
>> build time have the desired effect - being the ability to use HDFs dynamic
>> loading filters when interacting with netCDF 4 files?
>>
>> I hope my questions make sense - I’m feeling my way about THREDDS, NetCDF4
>> and HDF5 all at the same time.
>>
>> Thanks in advance
>>
>> Adam
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> thredds mailing list
>> thredds@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>
>
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/



-- 
Michael McDonald
Florida State University



  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: