[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Point data CDF format

Hi Bruce:
For station data, use the "contiguous list" option if you can. Its quite efficient and allows variable number of obs per station. If you only have one station, you can just call if a "point observation collection" and you dont need any linked list at all. 

Bruce Flynn wrote:

Hi John,
I did look at the Unidata Obs Convention and also what was available for the CF convention. Looking at the 2 different conventions was confusing me, but now I think I have a better understanding of what I was reading.
It looks like the Unidata standard says that you have to use a linked list of some sort. Because my main goal is to make the data easy to use for scientists. That being said, it would be really nice to be able to use the data in IDV, but I would rather not add the extra confusion of the linked list. Is there a way to make the data useful in IDV without using a linked list data structure? The Unidata standard seems to suggest that there isn't. 

Thanks
Bruce


On Dec 6, 2007, at 1:59 PM, John Caron wrote:


Hi Bruce:
CF "point obs" conventions are incomplete, and we dont have an adapter for them. as Don probably mentionaed, we are working on an improved version that we will submit to CF soon. in the meantime, if you need to get working with the IDV now, you should use the conventions at
http://www.unidata.ucar.edu/software/netcdf-java/formats/UnidataObsConvention.html 


Don Murray wrote:

Hi Bruce-

I'm going to pass this off to John Caron who knows the convention
better than I do. I don't think you want CF 1.0 as the convention
and I don't see that you have the  relation between the station
and the record.

John?

Don


Bruce Flynn wrote:

Don,
I now have a file that my CF validator says is valid and has the coordinate variable time. I tried to open it in IDV with the datatype of netCDF Point Data and it does not seem to work. If I leave the file chooser at "I'm feeling lucky" and select "netCDF Point Data" it fails silently. If I select "netCDF Point Data" from the start I get the following exception:
ucar.unidata.data.BadDataException: Unable to make a PointObsDataset from /Users/brucef/dev/cvs/TOOLS/dev/rig/python/db2nc/test.nc Error = **Failed to find Datatype Factory for= /Users/brucef/dev/cvs/TOOLS/dev/rig/python/db2nc/test.nc datatype= Station
at ucar.unidata.data.point.NetcdfPointDataSource.doMakeDataset(NetcdfPointDataSource.java:247) at ucar.unidata.data.point.NetcdfPointDataSource.getDataset(NetcdfPointDataSource.java:212) at ucar.unidata.data.point.NetcdfPointDataSource.initAfterCreation(NetcdfPointDataSource.java:199) at ucar.unidata.data.DataManager.createDataSource(DataManager.java:1222) at ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1829) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1748) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1682) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1663) at ucar.unidata.idv.chooser.IdvChooser.makeDataSource(IdvChooser.java:830) at ucar.unidata.idv.chooser.FileChooser.selectFilesInner(FileChooser.java:631) at ucar.unidata.idv.chooser.FileChooser.selectFiles(FileChooser.java:500) at ucar.unidata.idv.chooser.FileChooser.doLoadInThread(FileChooser.java:522) 
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
        at java.lang.reflect.Method.invoke(Method.java:585)
        at ucar.unidata.util.Misc$2.run(Misc.java:983)
        at ucar.unidata.util.Misc$3.run(Misc.java:1011)
I can't really tell from the exception if it's a problem with my data format or if it's a bug in the code.
Here's the latest NetCDF Header for my file, and I've also placed the file at ftp://ftp.ssec.wisc.edu/pub/incoming/test.nc if you want to take a look at it. 

netcdf test {
dimensions:
        time = UNLIMITED ; // (14851 currently)
        station = 1 ;
variables:
        double time(time) ;
                time:long_name = "Record epoch time offset" ;
time:units = "seconds since 1970-1-1 0:00:00 0:00" ;tation as calculated." ; 
                ACCURAIN:missing_value = -9999. ;
        double temperature(time) ;
                temperature:resolution = 5 ;
                temperature:units = "degC" ;
                temperature:standard_name = "air_temperature" ;
temperature:description = "Outside temperature reading." ; 
                temperature:missing_value = -9999. ;
        double rh(time) ;
                rh:resolution = 5 ;
                rh:units = "%" ;
                rh:standard_name = "relative_humidity" ;
rh:description = "Outside relative humidity reading." ; 
                rh:missing_value = -9999. ;
        double wspd(time) ;
                wspd:resolution = 5 ;
                wspd:units = "m/s" ;
                wspd:standard_name = "wind_speed" ;
wspd:description = "RM Young Wind Monitor speed reading." ; 
                wspd:missing_value = -9999. ;
        double wdir(time) ;
                wdir:resolution = 5 ;
                wdir:units = "degrees" ;
                wdir:standard_name = "wind_from_direction" ;
wdir:description = "RM Young Wind Monitor direction reading." ; 
                wdir:missing_value = -9999. ;
        double elevation(station) ;
                elevation:units = "meters" ;
elevation:description = "Height of the base of the RIG Tower" ; 
        double latitude(station) ;
                latitude:units = "degrees_east" ;
                latitude:description = "Latitiude of the RIG Tower" ;
        double longitude(station) ;
                longitude:units = "degrees_north" ;
                longitude:description = "Longitude of the RIG Tower" ;

// global attributes:
                :convention = "CF-1.0" ;
:description = "Meteorological Station Observations from the tower on the top of\n", "the UW SSEC/AOS building, commonly referd to as the Rooftop Instrument Group (RIG)." ; 
                :contact = "University of Wisconsin\n",
                        "Space Science and Engineering Center\n",
                        "1225 W. Dayton St.\n",
                        "Madison, WI 53706\n",
                        "http://www.ssec.wisc.edu"; ;
                :url = "http://rig.ssec.wisc.edu"; ;
}
On Dec 4, 2007, at 10:45 AM, Don Murray wrote:


Hi Bruce-

I guess the docs have been updated as we get ready to move to
Netcdf 4.  However, the CDL at:
http://www.unidata.ucar.edu/software/netcdf-java/formats/metar.cdl.txt 

doesn't seem to use structures but adheres to the convention.  I'm
cc'ing John Caron who is responsible for this convention for his input. 

Don

Bruce Flynn wrote:

Don,
Most of the record formats listed involve NetCDF Structures. I think this is obviously the best way to go for efficiency, but it seems there are not a lot of APIs that support reading Structures. I was planing on using the "pseudo-Structure" model so the scientists can use what ever API they want. It seems to want a station dimension though. Is this really necessary if you only have a single station? 
Bruce
On Dec 3, 2007, at 3:44 PM, Don Murray wrote:







--
*************************************************************
Don Murray                               UCAR Unidata Program
address@hidden                        P.O. Box 3000
(303) 497-8628                              Boulder, CO 80307
http://www.unidata.ucar.edu/staff/donm
*************************************************************