To clarify: the way I see it, you can do parallel I/O in three different ways.
The first is to reserve a process which will only deal with I/O and other
process will exchange data to read/write with it.
-- In a sense, this is not a parallel IO. It is using the sequential IO to
handle the parallel applications.
The second is to have each process read/write independantly.
-- You may talk about the independent IO here.
The third is to aggregate the I/O for several processes to improve performances.
-- You may mean collective IO here.
So my question was: in practice, which approach does parallel netCDF use ?
-- You can do both independent IO and collective IO with parallel HDF5.
Definitely collective IO for parallel NetCDF-3(argonne's parallel NetCDF) and
very also very possible independent IO.
> in strict performance terms -- which in the end is not really the
> be-all end all -- Argonne-Northwestern Parallel-NetCDF will be hard to
> beat, unless you are working with record variables.
Do you speak from personal experience ? I would be very interested in seeing
some data or benchmark about it.
-- There should be a paper that listed the flash benchmark comparison between
parallel NetCDF from Northwest(or parallel netcdf-3) and parallel HDF5.
However, it is an unfair comparison. It used collective IO for parallel
NetCDF-3 but independent IO for parallel HDF5. You can find more detailed
about the fair comparison with the collective IO for these two packages from
http://www.spscicomp.org/ScicomP12/Presentations/User/Yang.pdf
Be aware this was also a bit old. Don't know what's the current status between
these two packages.
Kent
--
Alexis Praga
_________________________________________________________
Ph.D Student
Aviation et Environnement
CERFACS
alexis.praga@xxxxxxxxxx
(33) 05 61 19 31 74
