Re: [netcdfgroup] simultaneous NetCDF writes to same file

Hi Kristopher,

> I am processing a large volume of satellite data where multiple
> processes could be simultaneously writing data to the same netcdf
> file.   This has not been supported in previous NetCDF versions and
> I've gotten fatal errors when two simultaneous writes conflicted.  I
> now understand that recent NetCDF versions do support this
> functionality.  Could someone tell me or provide an example of what I
> need to do (i.e. new
> function calls, options in netcdf open, etc...) to make this work for
> me?    I've tried the pnetcdf package does not support chunking which
> I need to internally compress these files.

No, sorry, it's not supported in current netCDF versions either.
NetCDF-4 uses HDF5 as its storage layer, and HDF5 does not support
compression with parallel access, as explained here:

  http://www.hdfgroup.org/hdf5-quest.html#p5comp

--Russ



  • 2012 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: