Richard P Signell wrote:
Is there a place on the web where we can find examples of 1D, 2D, and 3D
netcdf files that *do* work with IDV?
Hi Richard, I saw your posting. I am currently finishing a converter
from mm5v3 model output files to CF-1.0 compliant .nc files. A win is
direct consumption by IDV, release 1.2 at least. So far it has been
working ;)
If you load IDV 1.2 and go to Data Source Chooser|URLs and type
http://metoc.apl.washington.edu/uwme/2005090100-eta-d1.nc
then IDV should download a netcdf file from our server which includes 2D
and 3D model output fields.
If you wanted to see the .nc format, just type the same URL into a
regular browser and Save As to your local system and browse with ncdump.
The netcdf functionality of our server is somewhat experimental at the
moment, since I'm adding handling of the cross and dot staggering of mm5
grids, but at any time, the 'file' you get should be viewable in IDV.
I learnt most if not all about the CF conventions from the CF web page,
and from looking at the RUC.cdl example at ucar, and from answers from
Don Murray!
Also, I have learned that due to the way netcdf files are constructed,
IDV can actually ask for sections of data without needing all of the
file contents. Using the httpclient from Apache commons, IDV hits the
above url and in the http request says it wants bytes in the range say
0-20000. So our web server has to understand this (it's standard http
1.1, we use Tomcat 5.5.9). Think of this first data chunk as the table
of contents of a book, with page numbers to various chapters/sections.
Then when the IDV user wants to look at some field, IDV works out which
chapter (byte offset in fact) it's in from the TOC, and asks for a new
range. This is neat, since our mm5/nc files are ~50MB which could take
a while to transfer, while a single field/time combo may only be ~100K.
You can see all these range requests by (a) looking in the web server
access logs for 206 (partial content) rather than 200 (OK) records, and
by (b) using tcpmon (google it) as a middle man to view the http
messages back and forth. That's an invaluable tool I use it all the
time for http debugging. tcpmon becomes the client of the web site and
the server for IDV, forwarding bytes in both directions while also
displaying them in a swing panel set.
I will try to build a catalog (adde??) for our data set, at the moment
you have to know the url format, which is YYYYMMDDHH-model-nest.nc
where HH is 00 or 12, model can be eta, gfs (plus some others), domain
is "d1" or "d2". The data goes back to about 2001. See our web site at
http://metoc.apl.washington.edu/uwme/ for more details on what data we
are processing/offering up.
BTW if anyone is interested from a Java web app view, I use a servlet
filter technique to launch the mm5 to nc conversion at request time, and
session expiry to clear up local .nc files. Our mm5 data set is 3TB and
we don't have disk space for a local .nc copy as well, so we generate
just what users want ;)
I hope this rambling is all of use to the community, I had been meaning
to 'advertise' our .nc data anyways, so your post struck a chord. Feel
free to ask any questions...
Stuart
