[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: distributing mesonet data for use in IDV

John,
Congrats on getting the equipment grant, now you should have the disk space to store your data.
It's been awhile since our conversation at the AMS about serving up your surface obs using THREDDS and the IDV. Could you elaborate on how you expect it to work, what type of displays, and what type of software you are using. This would give us some idea on how to approach the problem. 

I did a prototype implementation of using the db4o database  to store the
METAR reports and to distribute them though the THREDDS server. But the db4o database had a too restrictive license so I dropped the effort. At this point, I don't know if any of that effort would help your effort. 


Thanks,
RObb...





On Fri, 8 Jun 2007, John Horel wrote:


Robb-
Got our letter yesterday saying we're funded from Unidata to move forward on the IDV hook to MesoWest. How have things progressed on your end as far as having the capability to query a surface ob database from within IDV? 

Regards,

John



Robb Kambic wrote:

On Wed, 14 Feb 2007, John Horel wrote:


Robb-
Starting to put together the equipment proposal to Unidata, which in part, would include a data server to serve up mesonet observations. So, I'd like to follow up on our conversations in San Antonio as far as possible approaches. I'm still a bit fuzzy as to what would be the best way to do it. My preference would be to develop a query directly to our database rather than having to store the data in an external hosted format. That way we don't have to keep around all 10 years of data in a netCDF file format or some such. Have you had a chance to proceed with your middleware to handle metars? And, could you describe that approach a bit more for me? 

Regards,

John



John,
your timing is perfect, yesterday our group had a meeting on this. in fact, the last month i created a realtime database to store the Metars. at this time there is a simple url servlet interface that permits one to make queries against it. so your idea of maintaining the data in a database is the correct approach. i'm just starting on making a general station observation dataset adapter that would create the link from some data repository ie database into the Java netcdf library, then the IDV. the station observation dataset adapter would get enough info so it could know how to query the proper data repository. at this time, i don't know the type/amount of info needed. i'll keep you informed on our progress.
do you keep all the data in one database? i was planning on daily creating a new database file for data that was 3 days old and only keeping ~3 days of data in the realtime database. this way the performance would be good for the realtime requests and archive requests would require opening the daily database files. 

Machine requirements guess...
the Thredds Data Server would have to be installed on the machine with cpu power to handle the amount of requests plus good network conectivity. i'm sure you have a better handle on the disk space requirements. you might want to look at THREDDS page for requirements for the TDS. 

http://www.unidata.ucar.edu/projects/THREDDS/

robb...
=============================================================================== Robb Kambic Unidata Program Center 
Software Engineer III               Univ. Corp for Atmospheric Research
address@hidden           WWW: http://www.unidata.ucar.edu/
=============================================================================== 



===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================