Introduction to Parallel I/O

21.1 Introduction to Parallel I/O

Parallel I/O allows many processes to read/write netCDF data at the same time.

Used properly, parallel I/O allows users to overcome I/O bottlenecks in high performance computing environments.
A parallel I/O file system is required for much improvement in I/O throughput.
NetCDF-4 (and pNetCDF) work with the Message Passing Interface, version 2 (MPI2). There are many software libraries which support MPI2. Any supercomputer will have some MPI2 library.
Other methods of parallelism are not supported by netCDF.
Parallel I/O can be developed on a Linux workstation, and parallel performace gains can be seen on multi-core machines. Machines will hundreds of processors can be simulated.
For netCDF testing we use the MPICH2 library.