NOTE: The netcdf-hdf mailing list is no longer active. The list archives are made available for historical reasons.
Hi Quincey,
1. Are there benefits to building HDF without --enable-threadsafe?As Orion mentioned, the C++ and FORTRAN wrappers aren't currently compatible with the threadsafe option. I don't think it would be too hard to address this, but it's probably enough work that we should look for some funding to do it.
I hope HDF finds the resources to keep the C and Fortran API at equivalent levels of functionality.
The only other downside really is the increased overhead for each HDF5 API call (to perform the semaphore locking that allows threadsafety).
Can you post or point me to numbers that quantify this?
If not, can you make it the HDF/netCDF4 default? at least on multi-core systems?That's probably difficult for us to detect at configure time. :-/
Yes, easier said than done. One might try imitating how packages like MPICH2 handle this.
We have learned, I think, to disable our progams' (NCO's) threading unless it is linked to threadsafe HDF. Otherwise users will experience unpredictable NCO failure. 2. How should we test, at NCO compile time, whether the underlying netCDF4/HDF install is threadsafe?The "H5_HAVE_THREADSAFE" macro will be defined when the "hdf5.h" header is included and threadsafety is enabled.
Good. This token will help us optimize NCO when built from source. I wanted our NCO programs to have threading available for general users but the configuration issues seem to dictate otherwise. For instance, we distribute NCO .debs and RPMs now with threading enabled by default because it "just works" with libnetcdf3.a. With netCDF4, however, pre-built binaries like debs and RPMs will depend on the distribution's libnetcdf4 .deb and RPM which likely will not be built on top of HDF --threadsafe. One outcome is that NCO users who rely on .debs and RPMs over custom installs will lose NCO threading on (at least) netCDF4 files. That is actually not a big deal since threading only speeds up math-heavy jobs (ncwa, ncap2) because I/O bottlenecks and threading overhead combine to slow down I/O intensive jobs (ncdiff, ncra,..). It's possible we may force OMP_NUM_THREADS=1 at run-time when input or output files are netCDF4 format, and only leave threading enabled for netCDF3 I/O in pre-built executables. Charlie -- Charlie Zender, Department of Earth System Science, UC Irvine Sab. at CNRS/LGGE-Grenoble until 20080815 :) 011+33+476+824236 Laboratoire de Glaciologie et Géophysique de l'Environnement 54 rue Molière BP 96, 38402 Saint Martin d'Hères Cedex, France
netcdf-hdf archives: