[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #HZY-708311]: ncdump/netCDF4 segfaults probing HDF4 file

Hi Charlie,

I created a Jira ticket for this issue, so you can subscribe to
be notified about any progress:

  https://bugtracking.unidata.ucar.edu/browse/NCF-273
  Provide a way to determine underlying file type for other than 
  the 4 netCDF-4 file types, such as DAP2, HDF5, HDF4, etc.

We also determined that HDF5 also has no easy way to determine, 
through an API, what features are enabled in the HDF5 library used
by an application.  However, they do make available a global variable,
'H5libhdf5_settings', which is a string that could be parsed to get
such information as is available in the 'libhdf5.settings' text file
that gets installed in the same directory as the library.

We haven't determined the best way to make such configuration-time
information available through an API, but will continue to consider
it.

Thanks for the suggestions.

--Russ

> Here are two issues and two suggestions to address them.
> 
> Issues:
> 1. NCO would like to support HDF4 files without requiring users to know
> that their files are HDF4.
> 2. NCO would like to continue to work on older netCDF libraries
> 
> Suggestions
> 1. Implement a netCDF4 API-based way of determining if a file is an
> HDF4 file at run-time. This could be error code returns, a function or
> a filetype mask. This will allow applications like NCO to know what
> is and is not possible to do with the file. e.g., netCDF API cannot
> be used to write directly to an HDF file. It would be helpful to know
> this automatically. Many users do not know what filetype their data
> is, so my temporary workaround in NCO, the --hdf4 switch, is not a
> user-friendly long-term solution.
> 
> 2. It is hard for applications like NCO to know whether the libnetCDF
> that it is linked to supports HDF4. If the libnetcdf.a does not
> support HDF4, and NCO knew that, then it could print helpful
> error/hint messages when things failed. NCO woul like accesse to a
> compile-time test that shows whether HDF4 is built into libnetcdf.
> This could be a new header token in netcdf.h. Or something else.
> Maybe the way #1 is implemented could also solve #2.
> 
> Thank you for considering both of these issues.
> 
> c
> 
> Le 07/10/2013 08:25, Unidata netCDF Support a écrit :
> > Hi Charlie,
> >
> >> This is an HDF4 file containing NASA MODIS snow cover data:
> >> http://dust.ess.uci.edu/hdf/MOD10CM.A2007001.005.2007108111758.hdf
> >>
> >> ncdump, compiled with --enable-hdf4, and the netCDF git snapshot from
> >> today, correctly dumps the metadata of the variable
> >> Snow_Cover_Monthly_CMG _unless_ the -s option is given:
> >>
> >> ncdump -s -v Snow_Cover_Monthly_CMG MOD10CM.A2007001.005.2007108111758.hdf
> >> (segfault)
> >>
> >> ncks also dies while trying to dump this metadata.
> >> ncks dies while calling nc_inq_var_chunking().
> >> I think ncdump dies in that or a similar function, i.e., a function
> >> trying to obtain netCDF4-style information from the variable.
> >>
> >> Is this something that can/should/will be fixed upstream from NCO
> >> such as in the netCDF library or the HDF4 or HDF5 library?
> >
> > Yes, it's a bug in netCDF-4 that we should fix.
> >
> >> Or should NCO develop a workaround, e.g., not attempting "certain
> >> functions" on HDF4 files? If so, which netCDF functions do not
> >> work on HDF4 files? Or, alternatively, which functions does netCDF4
> >> with --enable-hdf4 support on HDF4 files? And is there any detailed
> >> documentation on what does/doesn't from netCDF on HDF4 files?
> >
> > You shouldn't have to develop a workaround.  NetCDF functions that don't
> > work on HDF4 files should return the same way as for netCDF classic format
> > files, either with an error or a sensible value, such as 0 for the number
> > of subgroups or user-defined types.
> >
> > There's not much documentation on how netCDF functions behave with HDF4
> > files, so we'll have to write some as we fix this problem.  I'm opening
> > 2 new issue tickets you can follow if you want to see progress on this
> > or make suggestions on what behaviour you want for specific functions:
> >
> >   https://bugtracking.unidata.ucar.edu/browse/NCF-272
> >   https://bugtracking.unidata.ucar.edu/browse/NCF-273
> >
> > --Russ
> >
> >> Thanks!
> >> c
> >> --
> >> Charlie Zender, Earth System Sci. & Computer Sci.
> >> University of California, Irvine 949-891-2429 )'(
> >>
> >>
> >
> > Russ Rew                                         UCAR Unidata Program
> > address@hidden                      http://www.unidata.ucar.edu
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: HZY-708311
> > Department: Support netCDF
> > Priority: Normal
> > Status: Closed
> >
> 
> --
> Charlie Zender, Earth System Sci. & Computer Sci.
> University of California, Irvine 949-891-2429 )'(
> 
> 
Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: HZY-708311
Department: Support netCDF
Priority: Critical
Status: Closed