Roy Mendelssohn wrote:
Hi John:
On your web pages, you have some very interesting suggestions on how
to use structures and linked lists within netcdf to store profile or
in situ data. I am wondering if this is being used by anyone on real
data sets, and there experience with access times etc. etc.
Any info appreciated.
Regards,
-Roy M.
Hi Roy:
The latest draft of that is at
http://my.unidata.ucar.edu/content/software/netcdf-java/formats/UnidataObsConvention.html
we still havent finalized it, however we are using it in the IDD metar
decoder, a sample file (50 Mb) can be downloaded at
http://motherlode.ucar.edu:8088/thredds/fileServer/station/metar/20050802_metar.nc
in this catalog:
http://motherlode.ucar.edu:8088/thredds/catalog.html
we have not yet done any timing on this, but we know that for our
current purposes (implementing StationObsDataset in CDM) it has good
response time when reading from local files (not so good for remote
files dure to latency). The queries we optimize for are "get all obs at
a station", and "get all stations in a bounding box". The metar file
above has 25 - 50 observations for each of 4000 stations. Each obs
probably requires a disk access / network round trip. If you are writing
these files with complete control over the layout, id use the contiguous
list layout. (That assumes you want to optimise the "get all obs at a
station".)
we havent yet written or tested files for trajectory or profile data
yet, but we will.
let me know what plans you have, perhaps we can work together to
finalize this and get some tests run.
John
