On 11/07/2014 07:13 AM, Ed Hartnett wrote:
Howdy all!
Before Ward's time, I was the one who split out the Fortran libraries in
netCDF.
Although this represents some extra inconvenience for Fortran
programmers who want to build from source, it has a couple of benefits.
1 - Users who only want the C library don't get confused with the
Fortran build. This was a significant problem when the releases were
combined. Although the netCDF Fortran community is strong, there is an
even larger set of tools and APIs that only need the C library. The
Fortran build generated many more support questions than the C build. By
providing a C-only library distribution we cut support questions pretty
dramatically.
So, you orphaned us poor Fortran dinosaur programmers ...
... too bad because besides the (nice, high quality)
tools that are based in NetCDF-C, virtually all ocean/atmosphere/climate
models (and probably more software out there) is written in Fortran (of
various generations).
How could the more expert programmers of those tools get more confused
than we dinosaur Fortranists did after Fortran was dismembered from
NetCDF? :)
2 - Splitting the libraries by language fits a lot better with the
GNU/Linux worldview, and makes them easier and more attractive for
dedicated volunteers to distribute via the Linux package management
tools.
The argument isn't so strong, because the Fortran libraries are
hierarchically dependent on the C ones (and in a dependence chain).
Besides, the releases became totally independent, and who knows which
Fortran library versions are consistent with such and such C library
versions?
The build became much more cumbersome.
On my stock Fedora machine I can type "yum install
netcdf-fortran" and I will get the fortran build, and all the libraries
it depends on.
Not everybody wears Fedora.
RHEL/CentOS/SL, which are the stable distributions used
in many servers and clusters are ages late w.r.t. NetCDF packages
(4.1.1 I think, from Epel).
Not sure about the situation with SLES, Debian/Ubuntu, and friends.
But more important, packages are not always the best fit,
actually the worst in many cases.
Many programs depend on having libraries built with commercial
compilers (eg. Intel, PGI, etc), won't link programs otherwise.
So, one must have the ability to build from source with the compilers
of choice, to have this flexibility.
I (and many others) go through the trouble of building zlib, HDF5, the
whole chain of NetCDF dependencies in order to provide
a consistent and rich
environment for compilation and linking.
And on top of this build also some of the tools in the NetCDF
constellation (NCO, CDO, etc).
That abiltiy is becoming hindered by the disconnected versions of
the various NetCDF library interfaces distributed to the public.
That can be easily remedied without hindering the developers
preferences, by bundling them all into public releases with an
easy to use "configure/make/make-install" build system.
Please, don't throw the baby with the bath water.
Have you tried to compile NCAR, GFDL,
MaxPlanckInstitute (ECHAM and friends),
and other models with gfortran, and link them with NetCDF from packages?
Please, give the a try ...
F90 modules are not compatible across compilers, and if one uses
NetCDF packages there is no game when the programs require commercial
compilers.
This is by far the easiest way to install Netcdf, and it
does not generate support questions.
Easy, sure.
Effective, it is not, at least not for many applications.
NetCDF libraries are ported to most
of the mainstream Linux distros I've encountered, and even on Windows
with Cygwin. (But to use the Microsoft development tools you must use
Ward's CMake-based windows build.)
Best advice is to build from source only if you need the very latest
release, and it is not yet offered by your package management system.
Not necessarily for the latest release.
We can live with versions that are a tad old, but we need
flexibility to build from source because not all programs will work
otherwise.
Otherwise use the package management system and you will get all
dependencies and everything properly installed in the right place,
without having to read documentation or install multiple packages by
hand. I note that the Fedora distro helpfully includes versions of
netCDF built for parallel I/O as well. With all these helpful
distributions, the average netCDF user should not have to build from source.
Thanks,
Ed
Thanks,
Gus Correa
On Thu, Nov 6, 2014 at 8:55 PM, Gus Correa <gus@xxxxxxxxxxxxxxxxx
<mailto:gus@xxxxxxxxxxxxxxxxx>> wrote:
Thank you, Ward.
On 11/06/2014 06:06 PM, Ward Fisher wrote:
Thanks all for the input re: bundling the different interfaces; it’s
clear this is more convenient. I would argue that the benefit is not
strictly to us developers at the expense of the poor users (as
somebody
put it :) ); the split makes it much easier to provide support for
individual interfaces, as well as faster bug fixes as previously
mentioned.
There are significant technical hurdles to recombining the
interfaces
into a single project, as it was for versions 4.1.3 and prior.
However, the various interfaces have a hierarchical dependence,
all of them - except for Java - depending on the C interface.
This gives them some sort of unity, at least when one has to *use* them.
Nobody can get the F90 interface to work without the Fortran (77) one,
and the latter doesn't work without the C interface either.
Hence, from the user/programmer standpoint, I think breaking this
unit into smaller pieces is artificial and not helpful.
Additional difficulties happen also when one has to build more tools
in the NetCDF universe (NCO, NCL, CDO, etc) which depend on NetCDF
(and sometimes on specific library versions).
Those difficulties may also be reduced by having a clear and unified
public distribution.
However, I guess there may be a way to reconcile the two,
keeping the developement in separate projects,
and yet provide a bundled distribution for
users/programmers as it was before.
There may
be avenues for making distribution more transparent and easier
to keep
track of from the end user point-of-view, however. I may start a new
thread once I’ve explored a couple of ideas.
That would really be great for NetCDF users/programmers,
and hopefully won't be a hurdle for the developers.
As others list subscribers certainly did, I built NetCDF C 4.3.0,
along with Fortran 4.2 and C++4.4.2,
which are available from Unidata as separate tarballs.
I keep them under a single environment module in our clusters (actually
one per compiler), which is very convenient from the user point of view,
and seems to be all we need to compile and run correctly a
large number of programs (from NCAR, GFDL, MPII, MIT,
some inhouse codes, etc).
But what took some time for me to setup, may not be so
hard for the developers.
So, maybe Unidata could bundle the three main interfaces
into a single public/external/vanilla distribution,
perhaps under a single release name
(NetCDF.all.X.Y.Z perhaps?), which would build from a single
configure/make/make_install action, as it was done until 4.3.1,
and likewise for subsequent public releases.
The existence of a single bundled release also reassures the end user
that he is dealing with a consistent software package.
(S)he doesn't need to go crazy to find out what set of specific
interface releases are consistent with each other, or to spend too
much time building/testing each one separately.
Anyway, I'm glad that somehow my hijacking of this thread
was not just an annoyance to the list,
but had also a productive side.
Thank you
Gus Correa
Regarding the question below about binary distributions for OSX and
Windows. We provide Windows binaries because, frankly, building with
Visual Studio can be a bit of a mess, and providing the libraries
packaged with dependencies seemed like the easiest way to head
off a lot
of problems. These can be downloaded here:
*
http://www.unidata.ucar.edu/__software/netcdf/docs/winbin.__html
<http://www.unidata.ucar.edu/software/netcdf/docs/winbin.html>
I wasn’t under the impression that there was much need for OSX
binary
distributions, since OSX is essentially BSD and works with autotools
and/or CMake. I know that the popular package managers
|homebrew| and
|macports| have netcdf packages (which we do not maintain), and had
always thought these must be sufficient, as nobody has said
otherwise.
I’d be really interested to know if these were insufficient!
Thanks all,
-Ward
On Thu, Nov 6, 2014 at 3:34 PM, Chris Barker
<chris.barker@xxxxxxxx <mailto:chris.barker@xxxxxxxx>
<mailto:chris.barker@xxxxxxxx <mailto:chris.barker@xxxxxxxx>>__>
wrote:
On Thu, Nov 6, 2014 at 12:57 PM, Gus Correa
<gus@xxxxxxxxxxxxxxxxx <mailto:gus@xxxxxxxxxxxxxxxxx>
<mailto:gus@xxxxxxxxxxxxxxxxx
<mailto:gus@xxxxxxxxxxxxxxxxx>>__> wrote:
PS - Yes, I do understand the initial thread was about
NetCDF3 vs NetCDF4, which is a separate discussion in
itself.
Bringing it back ---
One of the main reasons users don't use netcdf4 is that it's
substantially harder to build (and more heavy weight). As
(at least
for the c libs) the netcdf4 libs fully support netcdf3,
there really
isn't any reason for all client code to use the netcdf4
libraries,
regardless of whether they are actually using netcdf4 files.
So anything Unidata can do to make it easier for end users
to use
netcdf4 will really help:
Easier to build
Good binary distributions (for Window and probably OS-X anyway)
???
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 <tel:%28206%29%20526-6959>
<tel:%28206%29%20526-6959> voice
7600 Sand Point Way NE (206) 526-6329
<tel:%28206%29%20526-6329> <tel:%28206%29%20526-6329> fax
Seattle, WA 98115 (206) 526-6317
<tel:%28206%29%20526-6317> <tel:%28206%29%20526-6317> main
reception
Chris.Barker@xxxxxxxx <mailto:Chris.Barker@xxxxxxxx>
<mailto:Chris.Barker@xxxxxxxx <mailto:Chris.Barker@xxxxxxxx>>
_________________________________________________
netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx <mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/__mailing_lists/
<http://www.unidata.ucar.edu/mailing_lists/>
