John Caron wrote:
>
>
> Greg Sjaardema wrote:
>> As a quick answer to the question, we (Sandia Labs) use netcdf
>> underneath our exodusII
>> file format for storing finite element results data.
>>
>> If the mesh contains #nodes nodes and #elements elements, then there
>> will be a dataset of the size #elements*8*4 (assuming a hex element with
>> 8 nodes, 4 bytes/int) to store the nodal connectivity of each hex
>> element in a group of elements (element block). Assuming 4GiB, this
>> limits us to ~134 Million elements per element block (using CDF-2) which
>> is large, but not enough to give us more than a few months breathing
>> room. Using CDF-1 format, we top out at about 30 million elements or
>> less which is hit routinely.
>
> Im not sure if I understand the problem yet:
>
> In the file you sent me, you use time as the record variable.
Yes.
>
> Each record variable must be less than 2^32, not counting the record
> dimension. So you can have about 2^29 elements, assuming each element
> is 8 bytes. And you can have 2^32 time steps.
Yes, but the record variables are currently not the limiting area.
>
> The non-record variables are dimensioned (num_elements, num_nodes).
> Number of nodes seems to be 8, and these are ints, so you have 32
> bytes * num_elements, so you can have a max of 2^27 elements = 134
> million elements.
Yes, this is where we are hitting the limit.
> Currently the largest you have is 33000. Do you need more than 2^27 ?
The file I sent was representative. As I think I mentioned in that
email, we have models ranging in size from 1 element up to 250 million
elements and everything in between. The header I sent was just to show
how the models are set up, not a model that is the largest we typically
use. Yes, we definitely need more than 2^27 elements.
>
> I think Im not sure what you mean by "a few months breathing room".
In a few more months if we do not change either the underlying netcdf
limits or the way that our exodusII format is using netcdf, we will have
to tell users that the models they require cannot be stored.
--Greg
>
>>
>> There is a pdf file at
>> http://endo.sandia.gov/SEACAS/Documentation/exodusII.pdf that shows
>> (starting at page 177) how we map exodusII onto netcdf. There have been
>> some changes since the report was written to reduce some of the dataset
>> sizes. For example, we split the "coord" dataset into 3 separate
>> datasets now and we also split the vals_nod_var into a single dataset
>> per nodal variable.
>>
>> --Greg
>>
>>
>> John Caron wrote:
>>> Hi Rob:
>>>
>>> Could you give use case(s) where the limits are being hit?
>>> I'd be interested in actual dimension sizes, number of variables,
>>> whether you are using a record dimension, etc.
>>>
>>> Robert Latham wrote:
>>>> Hi
>>>>
>>>> Over in Parallel-NetCDF land we're running into users who find even
>>>> the CDF-2 file format limitations, well, limiting.
>>>> http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/NetCDF-64-bit-Offset-Format-Limitations.html
>>>>
>>>>
>>>>
>>>> http://www.unidata.ucar.edu/software/netcdf/docs/faq.html#Large%20File%20Support10
>>>>
>>>>
>>>>
>>>> If we worked up a CDF-3 file format for parallel-netcdf (off the top
>>>> of my head, maybe a 64 bit integer instead of an unsigned 32 bit
>>>> integer could be used to describe variables), would the serial netcdf
>>>> folks be interested, or are you looking to the new netcdf-4 format to
>>>> take care of these limits?
>>>>
>>>> Thanks
>>>> ==rob
>>>>
>>> ==============================================================================
>>>
>>>
>>> To unsubscribe netcdfgroup, visit:
>>> http://www.unidata.ucar.edu/mailing-list-delete-form.html
>>> ==============================================================================
>>>
>>>
>>>
>>>
>
> ==============================================================================
>
> To unsubscribe netcdfgroup, visit:
> http://www.unidata.ucar.edu/mailing-list-delete-form.html
> ==============================================================================
>
>
>
==============================================================================
To unsubscribe netcdfgroup, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================
>From owner-netcdfgroup@xxxxxxxxxxxxxxxx 16 Wed, Sep
Date: Wed, 16 Sep 1992 22:48:00 EDT
From: RIVERS@xxxxxxxxxxxxxxxxxxx (Mark Rivers)
To: netcdfgroup@xxxxxxxxxxxxxxxx
Subject: IDL and PV-WAVE
Received: by unidata.ucar.edu id AA22873
(5.65c/IDA-1.4.4 for netcdfgroup-send); Wed, 16 Sep 1992 20:49:52 -0600
Received: from bnlx26.nsls.bnl.gov by unidata.ucar.edu with SMTP id AA22869
(5.65c/IDA-1.4.4 for <netcdfgroup@xxxxxxxxxxxxxxxx>); Wed, 16 Sep 1992
20:49:50 -0600
Organization: .
Keywords: 199209170249.AA22869
Message-Id: <920916224800.22800687@xxxxxxxxxxxxxxxxxxx>
X-Vmsmail-To: SMTP%"netcdfgroup@xxxxxxxxxxxxxxxx"
We are using a data visualization package called IDL from Research Systems Inc.
The nearly identical package is sold by Precision Visuals as PV-WAVE. The next
release of IDL will have built-in netCDF support.
For those IDL users who don't want to wait for the next release or for PV-WAVE
users, I have made the entire netCDF library callable from within IDL/PV-WAVE.
My version will only work directly on VAX/VMS but it will be rather simple to
port it to Unix. My interface exploits the ability of IDL/PV-WAVE to call
external routines written in C if those routines are linked into
sharable images (VMS) or shareable object libraries (Unix). Thus I linked the
netCDF library as a shareable image and just wrote a set of IDL "wrapper"
routines to call it.
The present version works fine as is. However, I will probably make changes to
the calling interface to match exactly the interface which IDL uses when it is
released.
If anyone is interested please let me know.
Mark Rivers (516) 282-7708 or 5626
Building 815 rivers@xxxxxxxxxxxxxxxxxxx (Internet)
Brookhaven National Laboratory rivers@bnl (Bitnet)
Upton, NY 11973 BNLX26::RIVERS (Physnet)
