Re: [netcdfgroup] NetCDF4 MPI-IO and compression

Rob Ross <rross@xxxxxxxxxxx> writes:

> In defense of the HDF team, this is difficult capability to enable in
> practice. This is because prior to compression you don't know how much
> space a given write will require, so parallelizing writes (especially
> if they aren't performed collectively) is not straightforward and
> requires a significant amount of coordination.
>
> Rob
>

Also I was mistaken when I wrote that parallel I/O and the deflate
filter can't be used together. This is only true for writing the data,
not reading it.

So you can read compressed data in parallel.

Thanks,

Ed
-- 
Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx


  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: