Actually, we developed conventions for doing this with netCDF about 8 y
ears
ago. I did post some information on the approach a couple of times in
the
past, but not in the last few years. The idea was supporting use of da
ta
in netCDF with Data Explorer (DX), since the DX data model supports man
y
more types of data than netCDF handles.
Here's a summary for your reference.
Data Explorer supports the importation of data in netCDF format, a data
abstraction for self-describing
multidimensional Arrays. It represents a simpler data model than that o
f
Data Explorer, one similar to that
of the Array Object. Data are accessed in netCDF through an application
programming interface
(available in C and FORTRAN libraries from the Unidata Program Center--
in
Boulder, Colorado).
Scalar data on a regular grid can be imported from a standard netCDF fi
le.
To import vector data, data on
irregular grids, or time series data, additional attributes must be add
ed
to the netCDF file. These attributes
allow you to specify the data, positions, and connections components of
your data set. See B.5 , "netCDF
Files: Complex Fields" for more information about these attributes.
Regular Grids
To import scalar data on a regular grid, specify the netCDF file name a
s
the name parameter. By default,
all netCDF variables are imported and collected into a Group. To import
one
or more particular variables,
specify their names as the variable parameter. The format parameter mus
t be
"netCDF."
Data Explorer automatically constructs positions and connections for ea
ch
variable, with an origin of 0.0
and spacings of 1.0 along each dimension.
For data that is logically a vector Field, but whose values are stored
in
three separate netCDF variables,
each component of the vector can be imported separately; the Compute mo
dule
can then be used to
create a single vector Field.
For data that is logically a vector Field, but whose values are stored
as
an n+1 dimensional regular grid,
use the Slice and Compute modules to separate the components of the vec
tor,
and then recombine them
into a single vector Field.
Example of a Regular Grid
The following file describes a 3 × 3 × 3 regular grid at origin 0,
0, 0
with deltas of 1.0 along each axis.
netCDF volume {
dimensions:
nx = 3;
ny = 3;
nz = 3;
variables:
float field_data(nx, ny, nz);
data:
field_data
0, 0, 0
0, 0, 0
0, 5, 0
0, 0, 5
0, 0, 0
0, 0, 0
5, 0, 0
0, 0, 0
0, 0, 0;
}
netCDF on completely regular grids can be imported directly by Data
Explorer without modifying the
netCDF file.
For data with more complex structure, conventions have been established for
netCDF variable attributes,
as described in the format below. The notation used corresponds to that of
the netCDL "language."
Irregular Arrays
This section describes how to specify netCDF variables for components with
irregular values.
Data
To indicate that a netCDF variable contains values corresponding to the
data component, it must have the
following attribute:
variable1:field = "fieldname";
Variable1 is the name of the netCDF variable containing data values to be
imported. fieldname is the
name of the Data Explorer field by which the user refers to the data (for
example, "temperature,"
"pressure," "wind"). If more than one variable is tagged with the same
field name, each variable is read into
a field, and the fields are collected into a group.
The data are read in as an array of values, one number per grid point. If
the data are actually a vector or a
matrix at each grid point, use one of the following modifiers:
variable1:field = "fieldname, vector";
variable1:field = "fieldname, matrix";
The non-scalar data are stored in additional dimensions for the variable.
For a static three-dimensional
3-vector, the three components are stored in a fourth dimension of size 3.
If the data have both regular connections and regular positions, no other
attributes are required. A regular
grid is assumed, with the origin at 0.0, and a spacing of 1.0 along each
axis. The number of axes will be
determined from the number of dimensions in the data array.
Positions
If the locations of the data values in variable1 do not form a regular
lattice (with origins at 0.0 and
spacings of 1.0), the name of a netCDF variable that contains the position
information must be specified as
an attribute for variable1.
There are five different types of position specifications: none, completely
regular, completely irregular, and
two types of partially regular.
Completely irregular is assumed if the following attribute is specified:
variable1:positions = "variable2";
where variable2 is an array of vectors, one for each grid point, defining
its location. The dimensionality of
the data space is determined by the number of items in a vector.
Regular positions can be specified with just the origin and spacing between
grid points along each axis in
compact form. The following attribute is used:
variable1:positions = "variable2, compact";
where variable2 is the name of a n
×2 array containing origin, delta pairs
for the spacing and location of
positions along each axis. The number of positions along each axis is
determined from the shape of
variable1.
Positions that can be specified as the product of arrays containing the
location of points along each axis
can be input in product form. Use the following attribute:
variable1:positions = "variable2a, product;
variable2b, product;
.
.
.
variable2x, product";
where the variable2's are each the name of an array containing a list o
f
positions along that axis. The
number of items in each array must match the length of the correspondin
g
axis in the original variable1
data array.
If any of the axes in an partially regular product array are actually
regular, they can be specified in
compact form:
variable1:positions = "variable2a, product, compact;
variable2b, product;
.
.
.
variable2x, product";
where variable2a is the name of an origin, delta array, and the rest ar
e
position lists as before.
Connections
If the connections between positions is a regular lattice, no additiona
l
attributes are necessary. For 1-D
data, connections of "lines" is assumed. 2-D data implies "quads," 3-D
data
implies "cubes" and for higher
dimensions, "hypercubes" is assumed.
If the connections are irregular, use one of the following attributes:
variable1:connections = "variable3, tetrahedra";
variable1:connections = "variable3, triangles";
variable1:connections = "variable3, cubes";
variable1:connections = "variable3, quads";
where variable3 is the name of an array containing a vector of point
numbers, defining each connection
element item. The length of this vector depends on the choice of
connections. If the shape is not explicitly
specified, tetrahedra are assumed.
Additional Components
If additional component information is present in the file, the followi
ng
attributes are valid:
variable1:component = "variable4, componentname, scalar;
variable5, componentname, vector;
variable6, componentname, matrix";
and
variable4:attributes = "ref, componentname;
dep, componentname";
Series Data
There are three ways to specify the import of datasets that should be
treated as series. They are:
Single variable
Separate variables
Separate files
Single Variable
When all data values are defined as a single netCDF variable, and the
unlimited dimension of the variable is
to be interpreted as the series dimension, then use one of the followin
g
forms of the field attribute:
variable1:field = "fieldname, scalar, series";
variable1:field = "fieldname, vector, series";
variable1:field = "fieldname, matrix, series";
All other specifications are the same as for simple fields.
The position and connection information is assumed to be constant for a
ll
members of the series. If the
positions or connections change for each step of the series, then the
variables used for those arrays must
also have an unlimited dimension that corresponds one-for-one with the
data
array.
An example using this method is provided in "Partially Regular Grids an
d
Time Series".
Separate Variables
When there are separate netCDF variables defined for each step in the
series, but all variables are in the
same file, use the following global attribute tags:
:seriesxxx = "fieldname;
variable1a;
variable1b;
.
.
.
variable1x";
or
:seriesxxx = "fieldname;
variable1a, float_value;
variable1b, float_value;
.
variable1x, float_value";
where the global tag must have the first 6 characters series. Global ta
gs
must be unique, so additional
characters can be added to distinguish them.
Each variable1x is the name array containing the data for that step. In
the
first format, the spacing of the
steps is assumed to be 1.0. In the second format, the float_value is th
e
value of each step. All other
specifications are the same as for simple fields.
Separate Files
When there are netCDF variables in separate files that make up the step
s of
a series, use the following
global attribute tags:
:seriesxxx = "fieldname, files;
filename1;
filename2;
.
.
.
filenameN";
or
:seriesxxx = "fieldname, files;
filename1, float_value;
filename2, float_value;
.
.
.
filenameN, float_value";
where the global tag must have the first 6 characters series. Global ta
gs
must be unique, so additional
characters can be added to distinguish them.
Each filenameN is the name of the netCDF file that contains the data
variables for that step. In the first
format, the spacing of the steps is 1.0. In the second format, the
float_value is the value of each step. All
other specifications are the same as for simple fields.
This format can be used to create short term series within a file, and
then
have a series of these smaller
series.
Examples
This section shows examples of netCDF files in the netCDL description
language. See the documentation
supplied by UCAR for more information on netCDL and the ncgen and ncdum
p
utilities.
Compact Specifications of Regular Dimensions
This example describes a single two-dimensional scalar field on a
latitude-longitude, regular, rectangular
grid. The example data are temperature on a one-degree grid with global
coverage. Because Data
Explorer array objects can be specified compactly, you can use this met
hod
to specify a netCDF with
regular dimensions. For each dimension, you need to specify its value a
t
the origin and its spacing along the
dimension.
In this example, two variable attributes are defined for the netCDF
variables. field specifies the rank of
the parameter, and positions specifies where the information containing
the
locations of the data is
space is located.
dimensions:
lon = 360;
lat = 180;
naxes = 2;
ndeltas = 2;
variables:
float locations(naxes, ndeltas);
float temperature(lat, lon);
temperature:field = "temperature, scalar";
temperature:positions = "locations, regular";
data:
locations = 89.5, -1., // compact specification, origin and
-179, 1.; // spacing for lat and lon
temperature = ... // Data for temperature
Partially Regular Grids and Time Series
This example describes an ocean circulation model that consists of a ti
me
series of four three-dimensional
scalars (temp, sali, wata, and conv) and one three-dimensional 3-vector
(vel). netCDF typically
requires seven variables, all scalars (the vector counting as three
scalars). The coordinate system for the
velocity vectors corresponds to that of the grid (that is, +u implies
north, +v implies east, and +w implies
down).
These grids are partially regular in that the time, tlat, and tlon port
ions
(three out of the four
dimensions) are all regularly spaced. time is to be mapped to members o
f a
series group. The fourth
dimension, tlvl, is irregularly spaced. The compact notation can be use
d
for the regular notation, while
the all values along the irregular dimension must be specified; a produ
ct
is formed from the dimensions.
Here is the specification in netCDL notation:
dimensions:
time = UNLIMITED;
tlat = 30;
tlon = 50;
tlvl = 30;
vsize = 3; // At each grid cell for variable vel, there are
// three floats for the u, v, and w components of the
// vector field.
naxes = 3;
ndeltas = 2;
variables:
float lat_axis(ndeltas, naxes);
float lon_axis(ndeltas, naxes);
float level_axis(tlvl, naxes);
float temp(time, tlat, tlon, tlvl);
temp:field = "temperature, scalar, series";
temp:positions = "lat_axis, product, compact; lon_axis,
product, compact; level_axis, product";
float sali(time, tlat, tlon, tlvl);
sali:field = "salinity, scalar, series";
sali:positions = "lat_axis, product, compact; lon_axis,
product, compact; level_axis, product";
float wata(time, tlat, tlon, tlvl);
wata:field = "water parage, scalar, series";
wata:positions = "lat_axis, product, compact; lon_axis,
product, compact; level_axis, product";
float conv(time, tlat, tlon, tlvl);
conv:field = "covective index, scalar, series";
conv:positions = "lat_axis, product, compact; lon_axis,
product, compact; level_axis, product";
float vel(time, tlat, tlon, tlvl, vsize);
vel:field = "velocity, vector, series";
vel:positions = "lat_axis, product, compact; lon_axis,
product, compact; level_axis, product";
data:
lat_axis = -14.667, 0., 0.,
0.333, 0., 0.;
lon_axis = 0.0, -99.8, 0.0,
0.0, 0.5, 0.0;
level_axis = 0.0, 0.0, 17.5,
0.0, 0.0, 53.425,
.
:
0.0, 0.0, 5374.98;
temp = ... ;
sali = ... ;
wata = ... ;
conv = ... ;
vel = ... ;
Irregular Surface
This example is the netCDL description of a netCDF for an irregular
surface, that of the classic teapot. It
has precomputed normals, which are imported as the "normals" component,
in
addition to positions and
connections.
netcdf teapot8 { // name of datafile is "teapot8.ncdf"
// name of field is "surface"
dimensions:
pointnums = 2268;
trinums = 3584;
axes = 3;
sides = 3;
variables:
float locations(pointnums, axes);
float normalvect(pointnums, axes);
long tris(trinums, sides);
float surfacedata(pointnums);
// global attributes:
:source = "Classic Teapot, data from Turner Whitted";
// specific attributes:
surfacedata:field = "surface";
surfacedata:connections = "tris, triangles";
surfacedata:positions = "locations";
surfacedata:component = "normalvect, normals, vector"
;
normalvect:attributes = "dep, positions";
// This is the start of a large data section
data:
·
}
More info about DX is available at www.ibm.com/dx
--------------------------
Lloyd A. Treinish
Visual Analysis
IBM Thomas J. Watson Research Center
P. O. Box 704
Yorktown Heights, NY 10598
914-784-5038 (voice)
914-784-7667 (facsimile)
lloydt@xxxxxxxxxx
http://www.research.ibm.com/people/l/lloydt/
Pascal Conreaux <stage6@xxxxxxxxxxxxxx> on 11/10/98 05:27:36 AM
Please respond to Pascal Conreaux <stage6@xxxxxxxxxxxxxx>
cc: (bcc: Lloyd A Treinish/Watson/IBM)
Hello everyone,
Netcdf is a good way to store a lot of different data.
But is there a way to specify (or an existing specification to learn)an
user that the Netcdf file describes, in 2D or 3D,
1)either a structured mesh
2)or an unstructured mesh
Thanks,
Pascal.
