hi, Richard and others
let's go back to my original example that produces the file "test_ds6.h5"
in /hl/test/test_ds.c (file is attached)
that illustrates a "typical" use of the Dimension Scales API for this case. I
had a set of bathymetry values for the North Atlantic (a 2D set of values) that
were used in a numerical hydrodynamic model *with variable grid size*. This
means that in the grid used, the squares that represent each cell are not
equal, some are bigger than others.
If you do a map in HDF Explorer by right clicking on the dataset named "data"
by choosing Discrete Map View, and then go to Options menu, choose Map, and
enable the check box that says "Grid", then you will see a map with a grid in
which the squares are different
This approach is commonly used in numeric models, where we want a finer
resolution in some areas, but not in others, because having a smaller grid for
all the domain might be computationally expensive.
So, anyway, this a perfect example of the use of HDF5 Dimension Scales. I
happened to have the spatial information along the X and Y axis (longitude and
latitude) values in ASCII files.
So, what that program does is to read all these ASCII files (data , lat and
lon) to memory and convert them to a DS HDF5 file
This is done in line 2691 of the file /hl/test/test_ds.c, for example for the
latitude
if(read_data("dslat.txt",1,latdims,&latbuf) < 0)
goto out;
then I make 3 HDF5 datasets (one with the lat values, a 1D array, another with
the lon values, a 1D array, another with my data values, the bathymetry, a 2D
array)
for example, for lat
if(H5LTmake_dataset_float(fid, "lat", 1, latdims, latbuf) < 0)
goto out;
note that in HDF Explorer, you see the "land" values (of Western Europe) in
grey color. That means that the data dataset was saved with a "fill value" for
the land values. The practical effect is that these values are not taken into
consideration when doing the legend and the map (in the legend, you will see
that the "sea" values range from 20 to 5500 meters) and the land values have
the value of -99. So the -99 value is ignored when doing the map and legend.
This dataset is made in line 2719 of test_ds.c. You will notice that this time
we did not use the High-Leval API. This is because the HL does not allow to use
fill values, so I used the basic API.
Then comes the part you will have to do for your datasets. You open the data
dataset, get an ID, DID variable here, open the latitude dataset, get its ID,
DSID variable here, and "link" the 2 with this call
if(H5DSattach_scale(did,dsid,DIM0) < 0)
what this function does is to associated the dataset DSID (latitude) with the
*dimension* specified by the parameter DIM0 (0, in this case, the first
dimension of the 2D array) of the dataset DID
If you open HDF Explorer and expand the attributes of the "data" dataset you
will see an attribute called DIMENSION_LIST.
This is done by this function. It is an array that contains 2 HDF5 references,
one for the latitude dataset, other for the longitude)
If you expand the "lat" dataset , you will see that it contains an attribute
called REFERENCE_LIST. It is a compound type that contains
1) a reference to my "data" dataset
2) the index of the data dataset this scale is to be associated with (0
for the lat, 1 for the lon)
this is also made by this function.
So, basically you just need to call this function H5DSattach_scale to link
together your scales with your data datasets
in HDF Explorer, if you open the "select dimension scale" dialog, there are
options to select the scales to anyone of the dimensions you have in your
dataset (you can switch lat and lon for example)
So, now to your file
I see that the dataset "lat" has the DIMENSION_LIST attribute. It should not
have, so I suspect this was somehow "incorrectly" linked, like I described
above. The scales should have the REFERENCE_LIST attribute.
I also see that your data (CH4, H2O, so on) is 1D and that it contains 1827
values (the values themselves seem very large, but my knowledge of CH4
concentration units is a bit rusty at this time -:) )
So, maybe you will want to link these datasets like this
H5DSattach_scale( CH4 dataset , lat , 0)
Also, your file exposed a bug in HDF Explorer (the 1D dataset use of dimension
scales was not checking for the 1D case) I uploaded a new version to the web
site. I get some strange values for the array indices though, for example for
CH4 I get all "1349.04" values in the indices, maybe a side effect of the DS
association you made in your file or just another bug in HDF Explorer
I don't quite understand your use of the corner coordinates of grid cells in
the lat_bnds
dataset, and the center coordinates to the grid cells in the variables lat and
lon, and also I don't have any suggestion of how "time" could be used here, but
I think now you have a better idea how to use our DS API
Regarding HDF View, I am not sure if there are plans to support Dimension Scales
The net-CDF folks might have some suggestions on the use of the "time" variable
here, since they regularly use it . By the way, netCDF4 also uses the HDF5
Dimension Scales API
some links
HDF Explorer
http://www.space-research.org/
Dimension Scales
http://www.hdfgroup.org/HDF5/doc/HL/
this latest page is being reworked now to include the Specification documents
for Dimension Scales (also for the Table and Image APIs). check it again soon
Pedro
a snaphot of HDF Explorer showing the "select dimension scale" dialog, and the
variable grid size on the map
Emacs!
At 04:03 AM 3/26/2008, Richard van Hees wrote:
>Dear Pedro,
>
>Many thanks for your quick response.
>
>The CF metadata conventions is a reference document for netCDF usage for the
>climate and forecast community
>(<http://cf-pcmdi.llnl.gov/>http://cf-pcmdi.llnl.gov/).
>
>As you have guessed correctly, the variables lat_bnds and lon_bnds are the
>corner coordinates of grid cells, while the variables lat and lon contain the
>center coordinates to the grid cells. From your answer, I understood that I
>have to make time, lat and lon dimension scales of the measured species (like
>H2O, CO and CH4). I am not sure that I should define four different datasets
>to define the corners of the grid cells (quite an ugly solution). Currently, I
>try to figure out how the netCDF4 library solves this case.
>
>Thank you for pointing me to HDF Explorer. Unfortunately, it only works under
>windows, while my main platform is Linux. I have access to a windows machine
>and the program works nicely on your product. I will use thios program to
>check my products. Will hdfview also be upgraded to work with DS?
>
>Again many thanks for your help.
>
>Best regards,
>
>Richard
>
>Pedro Vicente Nunes wrote:
>>
>>Dear Richard and others
>>
>>Barbara forwarded me your questions, since I implemented the Dimension Scales
>>(DS) API.
>>
>>By taking a look at your binary file I guess that lat_bnds and lon_bnds
>>contain 4 sets of 1826 latitude or longitude values.
>>
>>We did not preview this case of having several lat lon sets in one dataset.
>>That would be of course possible to do in the future but in the meantime and
>>if my assumption that those values are indeed 4 sets of lat and lon, you will
>>have to modify, if possible, the way you save these datasets
>>
>>For your program to "work" correctly with our DSs you will have to write each
>>set of these 4 sets on a different dataset (that is 4 datasets for lat and 4
>>for lon) , and then "link" each one of them with your data dataset
>>
>>You are right that there is no user's manual for this. I was hoping to write
>>such a guide in the near future by this was delayed for a couple months. It
>>will be done tough.
>>
>>In the meantime we only have the reference manual
>>
>>http://www.hdfgroup.org/HDF5/doc/HL/RM_H5DS.html
>>
>>and a test program for the API in the hdf5 distribution under
>>
>>/hl/test/test_ds.c
>>
>>by looking at this C program you'll probably have a guess about the "correct"
>>way of using it.
>>
>>Amongst other things, the program reads and writes realistic lat and lon data
>>from the North Atlantic
>>(I used this data from an earlier position I had in a numeric hydrodinamics
>>institute, IST , in Portugal)
>>
>>Also, do download this HDF5 vizualization program called HDF Explorer (if you
>>happen to use Windows)
>>
>>http://www.space-research.org/
>>
>>go the left menu and click download
>>
>>it handles dimension scales , you can add several and choose the ones you
>>want to be displayed in the map
>>
>>here's a snapshot and the HDF5 file that test program produces with the
>>Atlantic data is attached
>>
>>let us know if you have more questions and thanks for using the DS API
>>
>>what are the CF metadata conventions ?
>>
>>Pedro
>
>
>--------------------------------------------------------------
>Pedro
Attachment:
test_ds6.h5
Description: Binary data
netcdfgroup archives: