[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #YWP-403246]: compiling netCDF



Greetings all,

What version of the HDF5 lib does Tecplot use?

Thanks!

Sean

> Sean,
> 
> I followed your advice and compiled netcdf with hdf5 support, then
> used nccopy to upgrade a wrfout file from WRF to hdf5 format.  But,
> Steve Robinson at Tecplot has an hdf5 loader in Tecplot, and he is
> having trouble loading the converted data set.  He says the converted
> data file is actually hdf4 and not hdf5.  Could you please correspond
> with him to help solve this problem so that we can load wrfout files
> into Tecplot.   I cc'd him on this message so that you would have his
> e-mail address.
> 
> 
> Many Thanks,
> 
> Chad
> 
> On Fri, Jun 3, 2011 at 9:53 AM, Unidata netCDF Support
> <address@hidden> wrote:
> > Greetings Chad!
> >
> >> Sean,
> >>
> >> I just got HDF5 to support, but there is no "make install:" option.
> >
> > `make install` should work - did you follow the instructions at 
> > http://www.hdfgroup.org/ftp/HDF5/current/src/unpacked/release_docs/INSTALL 
> > regarding the ./configure setup?
> >
> >> Where do I put it so that when I compile netCDF again, it can find
> >> what it needs.  Can I just copy everything in the HDF5/bin folder to
> >> /usr/local/bin, or maybe I should make it /usr/local/hdf5/bin.  Are
> >> there any library or include files that need to go into
> >> /usr/local/hdf5?  I figure if I use a separate pat like
> >> /usr/local/hdf5/bin, and then add that to my $PATH or remove it from
> >> my $PATH, I should be able to compile netCDF with or without HDF5
> >> support.  It would be easier if there was an environmental variable
> >> that the netCDF compile script recognized, like
> >> export HDF5=/usr/local/hdf5.  Would that work?
> >
> > The configure script for netCDF4 includes an option --with-hdf5=<path to 
> > top level hdf5>. If you run the config script for hdf5 using --prefix=<path 
> > where you want hdf libs, bin, and include to go>, then simply use this path 
> > when you compile netCDF4. For example, you suggested /usr/local/hdf5 as a 
> > place to stuff hdf5. In that case, when configuring the build for hdf5, use
> >
> > ./configure --prefix=/usr/local/hdf5 [other options, as needed]
> >
> > Then, when compiling netCDF, use:
> >
> > ./configure --with-hdf5=/usr/local/hdf5 [other options, as needed]
> >
> >> That way I could
> >> enable or disable that variable for compiling netCDF with or without
> >> HDF5 support.
> >>
> >
> > Changing $PATH as needed should work for what you need - just make sure to 
> > keep close watch on what $PATH says because it can bite pretty hard.
> >
> >>
> > <paste>
> >>
> >> I do not know why /bin/sh points to dash by default. That is weird.
> >> I just tried an apt-get install bash, and it says I have the latest
> >> version. I double-checked this by typing
> >>
> >> chad@chad-MacBookPro:/bin$ whereis bash
> >> bash: /bin/bash /etc/bash.bashrc /usr/share/man/man1/bash.1.gz
> >>
> >> So, /bin/bash is the real bash. Should I change the symbolic link so
> >> that /bin/sh points to /bin/bash? Would that help? I do not know why
> >> /bin/sh points to dash instead of bash. Any idea? Seems stupid to
> >> me. I am going to go ahead and rm /bin/sh and point it to /bin/bash
> >> instead.
> >>
> >
> > The Ubuntu devs decided to make /bin/dash the default, and I am not sure of 
> > why they chose to do so. Does the Intel ifortvars.sh start with #!/bin/sh? 
> > If so, then that explains everything, as /bin/sh points to /bin/dash, and 
> > therefore /bin/dash was being used to run the script even though you are 
> > running /bin/bash.
> >
> >> I just tried your idea and I think it worked. When I ran a make
> >> -check, it says it passed all the tests. I will attach log files for
> >> you to double check.
> >
> > Things look good to me.
> >
> >>
> >> I do not have HDF5 compiled on this machine, because wrfhelp@ucar told
> >> me not to. But, I do have a compiled version of netCDF 363 (borrowed
> >> the binaries from a CentOS server at the university). So, I will have
> >> to keep both netCDF 4 without HDF5 for compiling WRF and doing WRF
> >> simulations and need to keep netCDF 4 with HDF5 suport, for using
> >> nccopy to convert the WRF output data into a format with HDF5 that
> >> Tecplot can read. As long a we can get netCDF to compile with HDF5,
> >> then I should be in good shape. Thanks for telling me about nccopy.
> >> Saves me a bunch of time so I do not have to write a custom data
> >> converter. Incidentally, I did ask the WRF people about their plans
> >> regarding support for netCDF4 with HDF5 support and they said its at
> >> least a year away.
> >>
> >> Now I need to re-compile it with HDF5. Can I use pre-compiled
> >> binaries, or should I compile HDF5 with the intel compilers to make
> >> sure that its fully compatible? There are linux 64 bit binaries of
> >> HDF5 available at
> >>
> >> http://www.hdfgroup.org/HDF5/release/obtain5.html
> >>
> >> I am leaning toward trying to compile HDF5 first. I know that WRF
> >> will not compile unless its compiled under the same OS with the same
> >> compilers. Is it the same for netCDF and HDF5?
> >>
> >> I am attaching logs of configure, make and make check. Tell me if the
> >> little trick in switching /bin/sh -> /bin/bash actually worked.
> >>
> >> I have one important question. I just compiled netCDF 4.1.2 without
> >> HDF5 support, and now want to compile WRF using it. If I have nccopy
> >> from a netCDF 4.1.2 compilation with HDF5 support, will nccopy still
> >> work, converting a WRF output file from 4.1.2 to 4.1.2 with HDF5
> >> support, or does nccopy only work for converting from netCDF 3.x to
> >> netCDF 4.x with HDF5 support?
> >>
> >
> > When you configured netCDF4, did you specify --enable-netcdf-4? If so, then 
> > I think nccopy will work.
> >
> > Sean
> >
> >
> >
> >>
> >> Thanks,
> >>
> >> Chad
> >>
> >> On Thu, Jun 2, 2011 at 2:47 PM, Unidata netCDF Support
> >> <address@hidden> wrote:
> >> > Chad,
> >> >
> >> >> Sean,
> >> >>
> >> >> I am using the atest ubuntu 11.0.4, just downloaded and installed las
> >> >> week.  Here is the ouput you requested:
> >> >>
> >> >> chad@chad-MacBookPro:~$ echo $SHELL
> >> >> /bin/bash
> >> >> chad@chad-MacBookPro:~$ ls -la /bin/sh
> >> >> lrwxrwxrwx 1 root root 4 May 25 21:07 /bin/sh -> dash*
> >> >> chad@chad-MacBookPro:~$
> >> >
> >> > The failures you get for `make check` show up when using ifort during 
> >> > the tests, and once again ifort is having issues finding the Intel 
> >> > shared objects. Just out of curiosity, is /bin/bash also a symlink to 
> >> > /bin/dash? If so, then that is the problem - dash is not 100% compatible 
> >> > with bash, and thus the Intel scripts may not be fully functional. If it 
> >> > is a link to dash, use apt-get to install bash and then try to rebuild.
> >> >
> >> >>
> >> >> I am not sure if I should open a separate support issue for this, but
> >> >> I am running into a data incompatibility problem.  I want to use
> >> >> Tecplot 360 (tecplot.com) for visualzing and making movies of output
> >> >> from WRF.  Tecplot 360 support HDF5 data, but WRF only suports netCDF
> >> >> without HDF5, from what I have read and been told by wrfhelp.  How can
> >> >> I get my data into Tecplot 360 from a netCDF file that has no HDF5?
> >> >> Do you know if WRF planps to upgrade to HDF5 support soon?
> >> >
> >> > I am not sure what the plans are with WRF,as it is not developed at 
> >> > Unidata. The folks at NCAR who are involved in the administration of the 
> >> > codebase should be able to answer that question (I'd check with 
> >> > WRFhelp). The questions to ask would be if/when WRF plans to use netCDF4.
> >> >
> >> >>
> >> >> I suppose we would need to get this netCDF 4 with HDF5 support to
> >> >> compile, then I could write a program that read the netCDF without
> >> >> HDF5 data, then wrote it back with netCDF with HDF5.  Is netCDF 4 with
> >> >> HDF5 backwards compatible with output from say netCDF 363?  So, if I
> >> >> write a code using netCDF4 with HDF5, can I read netCDF 363 data and
> >> >> then write it back in HDF5 format?
> >> >
> >> > netCDF4 will read data written with netCDF3 - no problems :-) That 
> >> > said.. .
> >> >
> >> >> Do you guys have any utiities that
> >> >> already do this?  A simple utility that reads in a netCDF 363 data
> >> >> file and upgrades it to netCDF4 with HDF5 support?  If you do not, you
> >> >> should write one.  It would be simple for you guys that know netCDF so
> >> >> well.   If I have to do it, its going to take me a while to figure out
> >> >> how.... time I really do not have right now.
> >> >
> >> > If you check out the netCDF FAQ 
> >> > (http://www.unidata.ucar.edu/software/netcdf/docs/faq.html), there is an 
> >> > entry about how to do this 
> >> > (http://www.unidata.ucar.edu/software/netcdf/docs/faq.html#fv13).
> >> >
> >> >> I can write code in
> >> >> Fortran and C, but would prefer to use Python.  Python can usually
> >> >> call C libraries, so do you know for sure if Python can link to
> >> >> netCDF4 with HDF5 support?
> >> >
> >> > Since nccopy can do what you need it to do (once we get it to compile, 
> >> > that is), it may be of general interest to know that there is a netCDF4 
> >> > python wrapper that is functional, although a few features are missing 
> >> > (see http://code.google.com/p/netcdf4-python/). I've compiled and used 
> >> > this on MacOSX and it works great so far.
> >> >
> >> > Sean
> >> >
> >> >> If you can think of any other solution to
> >> >> this problem, please let me know.
> >> >>
> >> >>
> >> >> Many Thanks,
> >> >>
> >> >> Chad
> >> >>
> >> >> On Wed, Jun 1, 2011 at 3:03 PM, Unidata netCDF Support
> >> >> <address@hidden> wrote:
> >> >> > Hi Chad,
> >> >>
> >> >>
> >> >>
> >> >> Iatest ubuntu 11.0.4, just downloaded and installed las week.  Here is
> >> >> the ouput you requested:
> >> >>
> >> >> chad@chad-MacBookPro:~$ echo $SHELL
> >> >> /bin/bash
> >> >> chad@chad-MacBookPro:~$ ls -la /bin/sh
> >> >> lrwxrwxrwx 1 root root 4 May 25 21:07 /bin/sh -> dash*
> >> >> chad@chad-MacBookPro:~$
> >> >>
> >> >> I am not sure if I should
> >> >>
> >> >>
> >> >> On Wed, Jun 1, 2011 at 3:03 PM, Unidata netCDF Support
> >> >> <address@hidden> wrote:
> >> >> > Hi Chad,
> >> >> >
> >> >> > It seems as though netCDF is still having issues finding the Intel 
> >> >> > libs. First, which version of Ubuntu are you running? Second, could 
> >> >> > you send me the output from the following two commands:
> >> >> >
> >> >> > 1) echo $SHELL
> >> >> >
> >> >> > 2) ls -la /bin/sh
> >> >> >
> >> >> > Thanks!
> >> >> >
> >> >> > Sean
> >> >> >
> >> >> >> Hi Sean,
> >> >> >>
> >> >> >> I did as you suggested.  I cleared all the shell environmental
> >> >> >> variables and used
> >> >> >> source /opt/intel/bin/iccvars.sh intel64
> >> >> >> source /opt/intel/bin/ifortvars.sh intel64
> >> >> >>
> >> >> >> The configure and make seemed to go ok, but when I ran "make check" 
> >> >> >> it
> >> >> >> failed 4 out of 5 of the tests.
> >> >> >>
> >> >> >> I made log files of each stage and attached them to this email for 
> >> >> >> you
> >> >> >> to examine.  Do you think gc/gortran would work better or stay with
> >> >> >> intel on ubuntu?  Others here at ASU reported better success with
> >> >> >> intel compilers, which is why I am using them.
> >> >> >>
> >> >> >>
> >> >> >> Thanks,
> >> >> >>
> >> >> >> Chad
> >> >> >>
> >> >> >>
> >> >> >> On Tue, May 31, 2011 at 1:00 PM, Unidata netCDF Support
> >> >> >> <address@hidden> wrote:
> >> >> >> > Greetings Dr. Coulliette!
> >> >> >> >
> >> >> >> >> Hello,
> >> >> >> >>
> >> >> >> >> My name is Dr. Chad Coulliette.  I am working with Professor Alex
> >> >> >> >> Mahalov at Arizona State University.  I have been trying to 
> >> >> >> >> compile
> >> >> >> >> netCDF on Ubuntu Linux 64-bit.  The hardware is a MacBook Pro 
> >> >> >> >> with a
> >> >> >> >> Intel Duo Core 64-bit processor.  I want to compile and use WRF, 
> >> >> >> >> but
> >> >> >> >> need netCDF first.  I have netCDF installed as a package using
> >> >> >> >> synaptic, but I suspect there is something deficient about it, so 
> >> >> >> >> we
> >> >> >> >> have been trying to compile and install netCDF from source code.  
> >> >> >> >> I
> >> >> >> >> have netCDF-4.1.2 and have tried with both GNU compilers and Intel
> >> >> >> >> compilers.  The GNU compilers seem to compile ok....no errors 
> >> >> >> >> during
> >> >> >> >> make, but when I run "make check" it finds problems, and the 
> >> >> >> >> message
> >> >> >> >> actually says to contact this email address.
> >> >> >> >
> >> >> >> > From looking through the make-gcc.log, you seem to be using the 
> >> >> >> > Intel fortran compiler `ifort` along side the gcc `cc` compiler - 
> >> >> >> > this is because .configure could not find any other fortran 
> >> >> >> > compiler (i.e. gfortran) in your $PATH. The error you are getting, 
> >> >> >> > as indicated in the make-test-gcc.log file, is due to problems 
> >> >> >> > finding the Intel shared objects. This is probably because when 
> >> >> >> > you building with a GNU environment in mind and so you didn't 
> >> >> >> > source ifortvars.sh. If you want to go the GNU route, try 
> >> >> >> > installing gfortran .
> >> >> >> >
> >> >> >> >> With the Intel
> >> >> >> >> compilers, it crashes during the make.  I included log files of 
> >> >> >> >> the
> >> >> >> >> configure, make and make check for the GNU compilers, and 
> >> >> >> >> included log
> >> >> >> >> files for the configure and make for the Intel compilers.
> >> >> >> >>
> >> >> >> >> For the Intel compiler, I have been following the instructions at
> >> >> >> >>
> >> >> >> >> http://software.intel.com/en-us/articles/performance-tools-for-software-developers-building-netcdf-with-the-intel-compilers/
> >> >> >> >>
> >> >> >> >> Are these the best compile instructions?  They might be too old,
> >> >> >> >> because I noticed a complaint about the compiler flag -xT being
> >> >> >> >> depreciated.  I tried to take it out and run everything over, but 
> >> >> >> >> the
> >> >> >> >> results were the same.
> >> >> >> >>
> >> >> >> >
> >> >> >> > The instructions above are for an older version of netCDF. I would 
> >> >> >> > check out the Unidata netCDF docs, which can be found here:
> >> >> >> >
> >> >> >> > http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-install/Building-on-Unix.html#Building-on-Unix
> >> >> >> >
> >> >> >> > These docs do not specifically look at using the Intel compiler, 
> >> >> >> > but it should give you some general reference.
> >> >> >> >
> >> >> >> > What happens if you do not set any flags by hand, but only source 
> >> >> >> > iccvars.sh and ifortvars.sh before building?
> >> >> >> >
> >> >> >> > Cheers!
> >> >> >> >
> >> >> >> > Sean Arms
> >> >> >> >
> >> >> >> >> Could you examine the log files and let me know what to do?
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> Thanks,
> >> >> >> >>
> >> >> >> >> Chad
> >> >> >> >>
> >> >> >> >>
> >> >> >> >
> >> >> >> >
> >> >> >> > Ticket Details
> >> >> >> > ===================
> >> >> >> > Ticket ID: YWP-403246
> >> >> >> > Department: Support netCDF
> >> >> >> > Priority: Normal
> >> >> >> > Status: Open
> >> >> >> >
> >> >> >> >
> >> >> >>
> >> >> >>
> >> >> >
> >> >> >
> >> >> > Ticket Details
> >> >> > ===================
> >> >> > Ticket ID: YWP-403246
> >> >> > Department: Support netCDF
> >> >> > Priority: Normal
> >> >> > Status: Open
> >> >> >
> >> >> >
> >> >>
> >> >>
> >> >
> >> >
> >> > Ticket Details
> >> > ===================
> >> > Ticket ID: YWP-403246
> >> > Department: Support netCDF
> >> > Priority: Normal
> >> > Status: Open
> >> >
> >> >
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: YWP-403246
> > Department: Support netCDF
> > Priority: Normal
> > Status: Open
> >
> >
> 
> 


Ticket Details
===================
Ticket ID: YWP-403246
Department: Support netCDF
Priority: Normal
Status: Open