On Sep 5, 2:54pm, Aubrey Holland wrote:
> Hello,
> I was just wondering if asynchronous io was supported by netcdf. My
> application is in need of a speed boost in the area of its NetCDf read.
> Thanks
>
> Aubrey Holland
> holland@xxxxxxxxxxxxx
>
>-- End of excerpt from Aubrey Holland
By default, the io in netcdf is asynchronous.
The implementation in netcdf-2.4 (libsrc/xdrposix.c) uses
a single buffer of size BIOBUFSIZ. This macro is set in libsrc/local_nc.h
and defaults to 8192.
The scheme in netcdf-3 is similar (libsrc/posixio.c), except that the
size is more dynamic. The library attempts to query the system as to
to the preferred block size using "struct stat" member "st_blksize". (If this
isn't supported on a system it just uses 8192.)
The size of the buffer is initialized to this, and may grow in units of this
amount, so that a single request from the higher layers may all fit in the
buffer. To protect "small" systems, the size of a single request is limited
by a compile time macro NC_PG_CHUNK, defined in libsrc/nc.h. This defaults
to 16384. This default is probably too small for real workstations. If you've
got lots of memory, you can bump it up accordingly.
In netcdf-3, you can turn off buffering (make all the writes synchronous)
by open'ing with the NC_SHARE flag.
Also, in netcdf-3, the io layer is much more isolated, making it possible to
implement your own in special circumstances. The io interface is defined by
libsrc/ncio.h. There are two implementations provided, libsrc/posixio.c
(usual unix system calls) and libsrc/ffio.c (uses CRAY ffio library calls).
Thank you for using netcdf.
-glenn
