Hi Greg:
Can you send me the output from ncdump -h on a typical file ?
How does the parallel part work? do certain sections of the file get reserved
for different nodes, or is there a master node that essentially serializes the
writing?
Greg Sjaardema wrote:
As a quick answer to the question, we (Sandia Labs) use netcdf
underneath our exodusII
file format for storing finite element results data.
If the mesh contains #nodes nodes and #elements elements, then there
will be a dataset of the size #elements*8*4 (assuming a hex element with
8 nodes, 4 bytes/int) to store the nodal connectivity of each hex
element in a group of elements (element block). Assuming 4GiB, this
limits us to ~134 Million elements per element block (using CDF-2) which
is large, but not enough to give us more than a few months breathing
room. Using CDF-1 format, we top out at about 30 million elements or
less which is hit routinely.
There is a pdf file at
http://endo.sandia.gov/SEACAS/Documentation/exodusII.pdf that shows
(starting at page 177) how we map exodusII onto netcdf. There have been
some changes since the report was written to reduce some of the dataset
sizes. For example, we split the "coord" dataset into 3 separate
datasets now and we also split the vals_nod_var into a single dataset
per nodal variable.
--Greg
John Caron wrote:
Hi Rob:
Could you give use case(s) where the limits are being hit?
I'd be interested in actual dimension sizes, number of variables,
whether you are using a record dimension, etc.
Robert Latham wrote:
Hi
Over in Parallel-NetCDF land we're running into users who find even
the CDF-2 file format limitations, well, limiting.
http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/NetCDF-64-bit-Offset-Format-Limitations.html
http://www.unidata.ucar.edu/software/netcdf/docs/faq.html#Large%20File%20Support10
If we worked up a CDF-3 file format for parallel-netcdf (off the top
of my head, maybe a 64 bit integer instead of an unsigned 32 bit
integer could be used to describe variables), would the serial netcdf
folks be interested, or are you looking to the new netcdf-4 format to
take care of these limits?
Thanks
==rob
==============================================================================
To unsubscribe netcdfgroup, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================
==============================================================================
To unsubscribe netcdfgroup, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================
