Hi,
I'm using netcdf4.1.3 (although 4.2 is also possible) in C. I currently have a
code to which I've added basic
netcdf4/hdf5 parallel support as opposed to the initial serial input. This
works well but I'd like to scale up and my
rather naive method may not work so well so I wondered if any experts could
voice their opinions...
I currently store my 3D data as a fixed length array of a compound type (x,y,z
co-ords and 2 items of data per point).
This is somewhat efficient as the data space is only about 4-5% populated and
therefore I can get away with only storing
an entry in the dataset for the populated points.
I'd like to change this to use some of the data-space dimensions as dimensions
in the dataset so that any code reading
the dataset doesn't have to do parsing and re-distribution of the data. Most of
the processing is done in chunks of
z-plane, so I thought I would use the z-dimension of the data as the primary
dimension of the dataset, define a variable
over this dimension and at each point have a variable length array (VLA) of a
compound type.
This would appear to be possible from the documentation but I cannot get my
head around how I would define this, I
couldn't find a suitable example to start from. Could anyone give me a pointer
on how to do this (or point me to an
example if there is one I'm missing)?
Also, is this likely to be wise with regards compression? I currently compress
the data using the deflate option which
gives me decent compression however I'm not sure if this would be sensible with
the new scheme.
Cheers,
-Nick.
--
Nick Johnson, Applications Developer, EPCC
2407 JCMB, King's Buildings, Mayfield Road, Edinburgh, EH9 3JF
e: Nick.Johnson@xxxxxxxx t: 0131 651 3388
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
