Michael,
I appreciate your interest in the details. Netcdf-3 does not include
support for "jagged arrays", so all data storage is in rectangular prisms.
However, there is some support for jagged arrays in netcdf-4.
--Dave
On Mon, May 11, 2015 at 3:13 PM, Michael Powell <mwpowellhtx@xxxxxxxxx>
wrote:
> On Mon, May 11, 2015 at 3:28 PM, Dave Allured - NOAA Affiliate
> <dave.allured@xxxxxxxx> wrote:
> > Michael,
> >
> > On Sun, May 10, 2015 at 2:35 PM, Michael Powell <mwpowellhtx@xxxxxxxxx>
> > wrote:
> >>
> >> It's been a few years since I've looked at netCDF at all, so this is a
> >> refresher. Also, this has no doubt been covered a million times, but I
> >> am confused about the nature of the data block.
> >
> >
> > You are inquiring specifically about netcdf-3 format (classic and 64-bit
> > offset), not netcdf-4 (HDF5), right?
>
> For now, yessir. Thanks for the follow up.
>
> > Unless you have some very good reason for attempting to access netcdf-3
> > format directly, please use the netcdf libraries (C, fortran, etc.) to
> read
> > and write. Rough knowledge of the low-level format is helpful for
> > understanding performance issues.
>
> It's good to know. Mostly, morbid curiosity, I enjoy working with file
> formats like this, but I did take note that there are C/C++ and other
> API readily available. I like to push my architectural skills over
> areas like this.
>
> >> If I understand the header/data parts correctly (which, I may not),
> >> non-record can be 'infinite'?
> >
> >
> > Not infinite, rather about 8 exabytes. But that is probably what you
> meant.
>
> Practically infinite, at any rate.
>
> > This is shown in a good summary table in the older netcdf documentation:
> >
> http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/Large-File-Support.html
> >
> >>
> >> Rather, the non-record, fixed-length
> >> parts should occur prior to the record data?
> >
> >
> > Yes. I think the following overview is a little less confusing than in
> the
> > BNF spec. Think of all netcdf-3 files as having five possible parts.
> All
> > parts except the header are optional. The two pads are a minor detail,
> > don't worry about them:
> >
> > * Header
> > * Header pad
> > * Fixed length variables
> > * Data pad
> > * Record data
> >
> >>
> >> How does one tell what is record versus non-record?
> >
> >
> > Many applications don't bother to look. Basic read access works the same
> > either way, except for performance issues with large files.
> >
> > If you care, the manual method is to look for a dimension labeled
> > "unlimited" in ncdump -h output. Any variables using this dimension are
> > record variables. All others are fixed length variables.
> >
> > There are inquiry functions which can do this under program control.
> >
> >>
> >> I think this is as a function of the var nelems, dimid, etc, but it's
> >> not especially clear from the BNF grammar, or associated verbiage. For
> >> example, what is meant by 'interleaved'? As a cross section of the
> >> specified variables?
> >
> >
> > Only record variables are interleaved. The sub-arrays ("cross sections")
> > for record subscript #0 of all record variables are combined together at
> the
> > start of the variable length section. The combined block for subscript
> #0
> > is often referred to casually as one "record". The combined block for
> > subscript #1 is the next record. And so on.
> >
> > Suppose you have precip(time, lat, lon) and time(time), and time(time).
> > Then the first record is precip(0,*,*) and time(0) together, and so on.
> >
> >> Also taking into account spiked variable array dimensions?
> >
> >
> > What do you mean by "spiked"?
>
> Jagged arrays: something like this (turned 90 degrees):
>
> ++
> ++++
> +
> ++++++
> ++++++
> ++
>
> And so on.
>
> >> I'm sure I'm barely scratching the surface here... Insights are welcome.
> >>
> >> Thank you...
> >>
> >> Regards,
> >>
> >> Michael
> >
> >
> > --Dave
>
