Re: [netcdfgroup] Proposed changes to netcdf-c library: Detecting netCDF versus HDF5

  • To: "dmh@xxxxxxxx" <dmh@xxxxxxxx>
  • Subject: Re: [netcdfgroup] Proposed changes to netcdf-c library: Detecting netCDF versus HDF5
  • From: Stephan Hoyer <shoyer@xxxxxxxxx>
  • Date: Tue, 29 Aug 2017 17:24:41 -0700
Bringing this back up again, now that I see that _NCProperties is included
in the documentation under "Attribute Conventions" [1].

Is there are a good way to handle cases where there is no valid netcdflib
version to include here, because the file wasn't written using netcdf-c?

What I've been doing is including h5netcdfversion=X.Y.Z in the
_NCProperties attribute instead of netcdflib=X.Y.Z. I'm thinking now that
I'd also like to include versions of other libraries used to generate the
file (e.g., h5py). Does that seem reasonable? Or should I rather drop
unexpected names altogether?

At the very least, I don't think I can include any sensible netcdflib
version.

[1]
http://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html

On Sat, Apr 30, 2016 at 12:45 PM, dmh@xxxxxxxx <dmh@xxxxxxxx> wrote:

> This will be documented in the existing documentation.
> At some point, it needs to be added to the netcdf NASA spec
> (https://earthdata.nasa.gov/standards/netcdf-4hdf5-file-format).
> =Dennis Heimbigner
>   Unidata
>
> On 4/29/2016 10:39 PM, Stephan Hoyer wrote:
>
>> What is the specification you're planning to use for the value of the
>> _NCProperties attribute? We'd like to able to generate compliant netCDF4
>> files with h5netcdf, which uses HDF5 directly to write netCDF4.
>>
>> I guess my main request is to not assume that only netCDF-C will write
>> _NCProperties attributes, and to ensure that netCDF-C and the specification
>> are flexible enough to allow for different implementations, maybe even
>> hypothetical implementations not using the HDF5 libraries at all.
>>
>> Thanks,
>> Stpehan
>>
>> On Fri, Apr 29, 2016 at 8:32 PM, dmh@xxxxxxxx <mailto:dmh@xxxxxxxx> <
>> dmh@xxxxxxxx <mailto:dmh@xxxxxxxx>> wrote:
>>
>>     I am about to commit a pull request for the netcdf-c library having to
>>     do with identifying the provenance and format of netcdf-4 files,
>>     and specifically targeted at detecting netcdf-4 files from HDF5 files.
>>
>>     This provenance consists of the following information. First, there is
>>     a hidden, persistent, attribute names _NCProperties.  It specifies the
>>     library versions of the netcdf library and the hdf5 library used to
>>     create the file. This attribute never changes during the lifetime
>>     of the
>>     file (unless modified deliberately thru the hdf5 API).
>>
>>     Second, there are two special, non-persistent, attributes that are
>>     computed from information already in the file.
>>     1. _SuperblockVersion
>>     2. _IsNetcdf4
>>     Non-persistence means these attributes do not actually appear in the
>>     file.  and are computed from other info already in the file.
>>
>>     The _SuperblockVersion attribute is a single integer giving the
>>     version
>>     number (currently 0-3) of the superblock in the hdf5/netcdf-4 file.
>>
>>     The _IsNetcdf4 attribute is a single integer 0/1 indicating if the
>>     file
>>     has various tags indicating it was produced thru the netcdf-4 API.
>>     This
>>     is computed by using the HDF5 API to walk the file to look for
>>     attributes specific to netcdf-4.  False negatives are possible for a
>>     small subset of netcdf-4 files, especially those not containing
>>     dimensions. False positives are (I think) only possible by deliberate
>>     modifications to an existing HDF5 file thru the HDF5 API. For
>>     files with
>>     the _NCProperties attribute, this attribute is redundant. For files
>>     created prior to the introduction of the _NCProperties attribute, this
>>     may be a useful indicator of the provenance of the file.
>>
>>     These three attributes are hidden in the sense that they can only be
>>     accessed thru the netcdf-C api calls via the name. They have no
>>     attribute number and will not be counted in the number of global
>>     attributes in the root group.
>>
>>     The simplest way to view these attributes is to use the -s flag to the
>>     ncdump command.
>>
>>     Comments are welcome.
>>
>>     =Dennis Heimbigner
>>     Unidata
>>
>>     _______________________________________________
>>     netcdfgroup mailing list
>>     netcdfgroup@xxxxxxxxxxxxxxxx <mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
>>     For list information or to unsubscribe,  visit:
>>     http://www.unidata.ucar.edu/mailing_lists/
>>
>>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
  • 2017 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: