Isn't netCDF utility "ncdump -k" to tell whether an HDF file has
all the required components understood by netcdf?
Wei-keng
On Mar 3, 2016, at 11:39 AM, Ed Hartnett wrote:
> This is a super interesting conversation for me! ;-)
>
> I must admit that when putting netCDF-4 together I never considered that
> someone might want to tell the difference between a "native" HDF5 file and a
> netCDF-4/HDF5 file. Well, you can't think of everything.
>
> I agree that it is not possible to canonically tell the difference. The
> netCDF-4 API does use some special attributes to track named dimensions, and
> to tell whether classic mode should be enforced. But it can easily produce
> files without any named dimensions, etc. So I don't think there is any easy
> way to tell.
>
> However, you can attempt to open a HDF5 file with nc_create, and, if it
> fails, use HDF5. (It should very rarely fail.)
>
> Ed
>
>
>
> On Wed, Mar 2, 2016 at 7:07 PM, Miller, Mark C. <miller86@xxxxxxxx> wrote:
> I like John's suggestion here.
>
> But, any code you add to any applications now will work *only* for files that
> were produced post-adoption of this convention.
>
> There are probably a bazillion files out there at this point that don't
> follow that convention and you probably still want your applications to be
> able to read them.
>
> In VisIt, we support >140 format readers. Over 20 of those are different
> variants of HDF5 files (H5part, Xdmf, Pixie, Silo, Samrai, netCDF, Flash,
> Enzo, Chombo, etc., etc.) When opening a file, how does VisIt figure out
> which plugin to use? In particular, how do we avoid one poorly written reader
> plugin (which may be the wrong one for a given file) from preventing the
> correct one from being found. Its kinda a hard problem.
>
> Some of our discussion is captured here. . .
>
> http://www.visitusers.org/index.php?title=Database_Format_Detection
>
> Mark
>
>
> From: Hdf-forum <hdf-forum-bounces@xxxxxxxxxxxxxxxxxx> on behalf of John
> Shalf <jshalf@xxxxxxx>
> Reply-To: HDF Users Discussion List <hdf-forum@xxxxxxxxxxxxxxxxxx>
> Date: Wednesday, March 2, 2016 1:02 PM
> To: HDF Users Discussion List <hdf-forum@xxxxxxxxxxxxxxxxxx>
> Cc: "netcdfgroup@xxxxxxxxxxxxxxxx" <netcdfgroup@xxxxxxxxxxxxxxxx>, Ward
> Fisher <wfisher@xxxxxxxx>
> Subject: Re: [Hdf-forum] Detecting netCDF versus HDF5
>
> Perhaps NetCDF (and other higher-level APIs that are built on top of HDF5)
> should include an attribute attached to the root group that identifies the
> name and version of the API that created the file? (adopt this as a
> convention)
>
> -john
>
> On Mar 2, 2016, at 12:55 PM, Pedro Vicente <pedro.vicente@xxxxxxxxxxxxxxxxxx>
> wrote:
> Hi Ward
> As you know, Data Explorer is going to be a general purpose data reader for
> many formats, including HDF5 and netCDF.
> Here
> http://www.space-research.org/
> Regarding the handling of both HDF5 and netCDF, it seems there is a potential
> issue, which is, how to tell if any HDF5 file was saved by the HDF5 API or by
> the netCDF API?
> It seems to me that this is not possible. Is this correct?
> netCDF uses an internal function NC_check_file_type to examine the first few
> bytes of a file, and for example for any HDF5 file the test is
> /* Look at the magic number */
> /* Ignore the first byte for HDF */
> if(magic[1] == 'H' && magic[2] == 'D' && magic[3] == 'F') {
> *filetype = FT_HDF;
> *version = 5;
> The problem is that this test works for any HDF5 file and for any netCDF
> file, which makes it impossible to tell which is which.
> Which makes it impossible for any general purpose data reader to decide to
> use the netCDF API or the HDF5 API.
> I have a possible solution for this , but before going any further, I would
> just like to confirm that
> 1) Is indeed not possible
> 2) See if you have a solid workaround for this, excluding the dumb ones,
> for example deciding on a extension .nc or .h5, or traversing the HDF5 file
> to see if it's non netCDF conforming one. Yes, to further complicate things,
> it is possible that the above test says OK for a HDF5 file, but then the read
> by the netCDF API fails because the file is a HDF5 non netCDF conformant
> Thanks
> ----------------------
> Pedro Vicente
> pedro.vicente@xxxxxxxxxxxxxxxxxx
> http://www.space-research.org/
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@xxxxxxxxxxxxxxxxxx
> http://secure-web.cisco.com/1r-EJFFfg6rWlpQsvXstBNTjaHQaKT_NkYRN0Jj_f-Z3EK0-hs6IbYc8XUBRyPsH3mU3CS0iiY7_qnchCA0QxNzQt270d_2HikCwpAWFmuHdacin62eaODutktDSOULIJmVbVYqFVSKWPzoX7kdP0yN9wIzSFxZfTwfhU8ebsN409xRg1PsW_8cvNiWzxDNm9wv9yBf9yK6nkEm-bOx2S0kBLbg9WfIChWzZrkpE3AHU9I-c2ZRH_IN-UF4g_g0_Dh4qE1VETs7tZTfKd1ox1MtBmeyKf7EKUCd3ezR9EbI5tK4hCU5qW4v5WWOxOrD17e8yCVmob27xz84Lr3bCK5wIQdH5VzFRTtyaAhudpt9E/http%3A%2F%2Flists.hdfgroup.org%2Fmailman%2Flistinfo%2Fhdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@xxxxxxxxxxxxxxxxxx
> http://secure-web.cisco.com/1r-EJFFfg6rWlpQsvXstBNTjaHQaKT_NkYRN0Jj_f-Z3EK0-hs6IbYc8XUBRyPsH3mU3CS0iiY7_qnchCA0QxNzQt270d_2HikCwpAWFmuHdacin62eaODutktDSOULIJmVbVYqFVSKWPzoX7kdP0yN9wIzSFxZfTwfhU8ebsN409xRg1PsW_8cvNiWzxDNm9wv9yBf9yK6nkEm-bOx2S0kBLbg9WfIChWzZrkpE3AHU9I-c2ZRH_IN-UF4g_g0_Dh4qE1VETs7tZTfKd1ox1MtBmeyKf7EKUCd3ezR9EbI5tK4hCU5qW4v5WWOxOrD17e8yCVmob27xz84Lr3bCK5wIQdH5VzFRTtyaAhudpt9E/http%3A%2F%2Flists.hdfgroup.org%2Fmailman%2Flistinfo%2Fhdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>
>
>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
