Taylor,
If you are just trying to learn how to use the NetCDF C++ library, then Corey
is right, MERRA in its original HDF-4 format is not going to do the trick, even
with augmenting which is meant for HDF-5, not MERRA's HDF-4 format. (However,
I believe the NetCDF-Java library CAN read it--adds to the confusion a bit I
agree, but also makes for some nice behavior in tools like Panoply.) Our data
center does have some HDF5/HDF-EOS5 data you can play with along those lines,
such as Ozone Monitoring Instrument data
(http://mirador.gsfc.nasa.gov/cgi-bin/mirador/presentNavigation.pl?tree=project&project=OMI).
HOWEVER, if your main interest is in using the MERRA data via the NetCDF C++
library, then you have a multitude of options. The MERRA data are available
through two OPeNDAP servers (Hyrax and GDS), which should be readable by the
netCDF library over the network via OPeNDAP. If that does not work for you,
then there is also on-the-fly conversion to netCDF at
http://disc.sci.gsfc.nasa.gov/daac-bin/FTPSubset.pl.
If you have any problems working with these resources, please contact our
help desk at gsfc-help-disc@xxxxxxxxxxxxxx.
On Jan 4, 2013, at 5:55 PM, Corey Bettenhausen <corey.bettenhausen@xxxxxxxxxx>
wrote:
> Taylor,
> HDF-EOS is not the same as HDF-EOS5. HDF-EOS is built upon HDF4 while
> HDF-EOS5 depends on HDF5. While HDF4 and HDF5 may seem similar, they are not
> and should basically be considered two different file formats. Your MERRA
> data is HDF-EOS and thus anything you use must be explicitly for HDF-EOS
> files or, more generally, HDF4 files.
>
> My first suggestion would be to simply use the HDF4 library
> (http://www.hdfgroup.org). If you just need to read SDS's and do something
> with them, this may suffice.
> Cheers,
> -Corey
>
> On Jan 4, 2013, at 5:19 PM, Taylor Binnington wrote:
>
>> Thank you Russ.
>> I've gone ahead and installed the HDF5 augmentation tool (called aug_eos5),
>> which also requires the HDF5 and HDF-EOS5 libraries, which I have installed.
>> Following from the readme that accompanies the aug_eos5 documentation, I
>> attempted to run the default augmentation:
>>
>> aug_eos5 some_MERRA_file.hdf
>>
>> I received the errors:
>>
>> HDF5-DIAG: Error detected in HDF5 (1.8.10) thread 0:
>> #000: EHapi.c line 1493 in HE5_EHopen(): File
>> "MERRA300.prod.assim.inst3_3d_asm_Cp.20120101.hdf" cannot be opened.
>>
>> major: File accessability
>> minor: Unable to open file
>> #001: ../../src/H5F.c line 1582 in H5Fopen(): unable to open file
>> major: File accessability
>> minor: Unable to open file
>> #002: ../../src/H5F.c line 1373 in H5F_open(): unable to read superblock
>> major: File accessability
>> minor: Read failed
>> #003: ../../src/H5Fsuper.c line 334 in H5F_super_read(): unable to find
>> file signature
>> major: File accessability
>> minor: Not an HDF5 file
>> #004: ../../src/H5Fsuper.c line 155 in H5F_locate_signature(): unable to
>> find a valid file signature
>> major: Low-level I/O
>> minor: Unable to initialize object
>> HDF5-DIAG: Error detected in HDF5 (1.8.10) thread 0:
>> #000: GDapi.c line 309 in HE5_GDopen(): Cannot open the file
>> "MERRA300.prod.assim.inst3_3d_asm_Cp.20120101.hdf". Check the file name.
>>
>> major: File accessability
>> minor: Unable to open file
>> error(3) at read.c:730 | HE5_GDopen
>> error(3) at run.c:1253 | open hdfeos5 file
>>
>> Since this was supposed to be a simple default test of the augmentation
>> tool, but it failed, I'm stuck as to where to go. I have tested on a few
>> MERRA hdf files, all produce the same result.
>>
>>
>> On Mon, Dec 17, 2012 at 1:18 PM, Russ Rew <russ@xxxxxxxxxxxxxxxx> wrote:
>> Taylor,
>>
>> The netCDF-Java library can read HDF-EOS data, so if the data is
>> available from a server that uses netCDF-Java, such as TDS, then your
>> C++ program can access the HDF-EOS data using the OPeNDAP client
>> that's available with a recent netCDF-C based library built with DAP
>> protocol support enabled (the default if libcurl is available at build
>> time).
>>
>> As Dan Packman pointed out, you could instead download and make use of
>> one of the HDF-EOS tools available from
>>
>> http://hdfeos.org/software/tool.php
>>
>> which also includes
>>
>> HDF-EOS5 to netCDF-4 Converter -- converts HDF-EOS5 to netCDF-4
>>
>> Description: HDF-EOS5 to netCDF-4 Converter converts an HDF-EOS5 file
>> to a netCDF-4 file that the netCDF-4 library can read. Unlike the
>> HDF-EOS5 Augmentation Tool, this tool creates a new file that the
>> HDF-EOS5 library cannot read.
>>
>> --Russ
>>
>>> Hi Taylor,
>>>
>>> it's been a few years since I analyzed HDF-EOS data, but in case it's
>>> useful, I'll offer that at that time HDF-EOS was a specialized kind of
>>> *HDF* file, not a specialized kind of netcdf file. You need a separate
>>> HDF-EOS library to access those files, not the netcdf4 library.
>>>
>>> Maybe things have changed, but you might be better off first looking
>>> for HDF-EOS specific libraries.
>>>
>>> Regards,
>>>
>>> --Dave
>>>
>>>
>>> On Mon, Dec 17, 2012 at 9:35 AM, Taylor Binnington
>>> <tbinnington@xxxxxxxxx> wrote:
>>>> Thank you for the replies.
>>>>
>>>> I see. I'm still learning the differences between these file formats. It
>>>> appears that MERRA climate data is stored in HDF-EOS format, which is a
>>>> specialized kind of NetCDF-4... what are the chances that I will ever be
>>>> able to read this, even with newest versions of NetCDF with some HDF5 and
>>>> HDF4 support?
>>>>
>>>> I am running netcdf-4.0.1-19.1, since that is what was available in the
>>>> openSUSE geo repository as an rpm (therefore no, I did not make check befor
>>> e
>>>> installing). I will uninstall this and manually install the binaries to tes
>>> t
>>>> the most current version (with fingers crossed that HDF-EOS is supported).
>>>>
>>>>
>>>> On Mon, Dec 17, 2012 at 11:56 AM, Russ Rew <russ@xxxxxxxxxxxxxxxx> wrote:
>>>>>
>>>>> Taylor,
>>>>>
>>>>> More recent versions of netCDF-4 can access HDF5 data (and even HDF4
>>>>> data) that was not written by netCDF-4, but not arbitrary HDF5 files.
>>>>> Interoperability with HDF5 is documented here:
>>>>>
>>>>> http://www.unidata.ucar.edu/netcdf/docs/interoperability_with_hdf5.html
>>>>>
>>>>> and in these FAQs:
>>>>>
>>>>> http://www.unidata.ucar.edu/netcdf/docs/faq.html#fv20
>>>>> http://www.unidata.ucar.edu/netcdf/docs/faq.html#fv15
>>>>>
>>>>> If netCDF-4 gets a segfault opening an HDF5 file, that's either a bug in
>>>>> the legacy C++ library you're using or a bug in the accessing program .
>>>>>
>>>>> I just tried this example C++ program with the C++ library in netCDF
>>>>> version 4.1.3:
>>>>>
>>>>> http://www.unidata.ucar.edu/netcdf/examples/programs/simple_xy_rd.cpp
>>>>>
>>>>> changing the test file to a file that doesn't exist, and
>>>>> it seemed to behave correctly:
>>>>>
>>>>> $ ./simple_xy_rd
>>>>> Couldn't open file!
>>>>>
>>>>> What version of the netCDF C++ library are you using? Did you run "make
>>>>> check" before installing it?
>>>>>
>>>>> --Russ
>>>>>
>>>>>> Hi Taylor:
>>>>>>
>>>>>> The netcdf library can not open arbitrary HDF files, only ones built
>>>>>> with the
>>>>>> netcdf4 library. If this file was not built with the netcdf4 library,
>>>>>> then
>>>>>> yes you will get an error.
>>>>>>
>>>>>> HTH,
>>>>>>
>>>>>> -Roy M.
>>>>>> On Dec 17, 2012, at 8:14 AM, Taylor Binnington <tbinnington@xxxxxxxxx>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> This is my first time using the NetCDF C++ libraries, and I can't seem
>>>>>>> to g
>>>>>> et started. I would like to open an HDF file from a program using
>>>>>> NcFile, usi
>>>>>> ng the following:
>>>>>>>
>>>>>>> NcFile dataFile("MERRA300.prod.assim.inst3_3d_asm_Cp.20120101.hdf",
>>>>>>> NcFile:
>>>>>> :ReadOnly);
>>>>>>>
>>>>>>> However, gdb reveals a segfault at this line (yes, the file is stored
>>>>>>> in th
>>>>>> e same directory as the test script). Interestingly, I also receive the
>>>>>> same
>>>>>> error when I intentionally attempt to open a file that does not exist,
>>>>>> which
>>>>>> I find strange (I would have expected a 'file not found' type of error),
>>>>>> and
>>>>>> it makes me think that I simply can not open an HDF file with these
>>>>>> libraries
>>>>>> . Is this true? How would I proceed? Thank you in advance,
>>>>>>>
>>>>>>> Taylor Binnington
>>>>>>>
>>>>>>> --
>>>>>>> Taylor Binnington
>>>>>>> e. tbinnington@xxxxxxxxx
>>>>>>> c. 647 926 4144
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> netcdfgroup mailing list
>>>>>>> netcdfgroup@xxxxxxxxxxxxxxxx
>>>>>>> For list information or to unsubscribe, visit:
>>>>>>> http://www.unidata.ucar.edu
>>>>>> /mailing_lists/
>>>>>>
>>>>>> **********************
>>>>>> "The contents of this message do not reflect any position of the U.S .
>>>>>> Governm
>>>>>> ent or NOAA."
>>>>>> **********************
>>>>>> Roy Mendelssohn
>>>>>> Supervisory Operations Research Analyst
>>>>>> NOAA/NMFS
>>>>>> Environmental Research Division
>>>>>> Southwest Fisheries Science Center
>>>>>> 1352 Lighthouse Avenue
>>>>>> Pacific Grove, CA 93950-2097
>>>>>>
>>>>>> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
>>>>>> voice: (831)-648-9029
>>>>>> fax: (831)-648-8440
>>>>>> www: http://www.pfeg.noaa.gov/
>>>>>>
>>>>>> "Old age and treachery will overcome youth and skill."
>>>>>> "From those who have been given much, much will be expected"
>>>>>> "the arc of the moral universe is long, but it bends toward justice"
>>>>>> -MLK Jr.
>>>>>>
>>>>>> _______________________________________________
>>>>>> netcdfgroup mailing list
>>>>>> netcdfgroup@xxxxxxxxxxxxxxxx
>>>>>> For list information or to unsubscribe, visit:
>>>>>> http://www.unidata.ucar.edu/m
>>>>>> ailing_lists/
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Taylor Binnington
>>>> e. tbinnington@xxxxxxxxx
>>>> c. 647 926 4144
>>>>
>>>>
>>>> _______________________________________________
>>>> netcdfgroup mailing list
>>>> netcdfgroup@xxxxxxxxxxxxxxxx
>>>> For list information or to unsubscribe, visit:
>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>
>>>
>>>
>>> --
>>> David W. Pierce
>>> Division of Climate, Atmospheric Science, and Physical Oceanography
>>> Scripps Institution of Oceanography, La Jolla, California, USA
>>> (858) 534-8276 (voice) / (858) 534-8561 (fax) dpierce@xxxxxxxx
>>
>>
>>
>> --
>> Taylor Binnington
>> e. tbinnington@xxxxxxxxx
>> c. 647 926 4144
>> _______________________________________________
>> netcdfgroup mailing list
>> netcdfgroup@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>
> --
> Corey Bettenhausen
> Science Systems and Applications, Inc
> NASA Goddard Space Flight Center
> 301 614 5383
> corey.bettenhausen@xxxxxxxxxx
>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
--
Dr. Christopher Lynnes, NASA/GSFC, ph: 301-614-5185
