Hi Arun:
Before i try to answer any of your questions, I have a couple of my own:
1. Does SRB have a "data model" of some sort? I was under the impression that it mostly moved files around. NetCDF, in contrast, allows clients to extract subsets of a file's contents.
2. What functionality does it provide in its data access protocol? Do you have
a brief summary that i could look at?
Arun Jagatheesan wrote:
John:
I have been requested to formulate a technical strategy for NetCDF and SRB.
The expected outcome is an interface or integration at the API or protocol
level or it could be an architecture using which users can take advantage of
both SRB and NetCDF. I don't really know the commitment or requirement on
this effort at this stage. However, I think it worth to get this started
(this has been on our list for a long time).
SRB is a data grid management system for distributed management of data. It
does not have its own data format. However it has it own protocol for data
access. I just started reading some information from NetCDF webpages and
also the java guide:
ftp://ftp.unidata.ucar.edu/pub/netcdf-java/v2.2.14/NetcdfJavaUserManual-2.2.
doc
Based on my current level of understanding (which is just based on browsing
NetCDF webpages), it seems I could use a two-phase approach:
Phase 1: Initial, get it started phase (developer friendly - short term)
Phase 2: More detailed infrastructure level (user friendly - long term)
In phase-1 I could add a SRB-IOServiceProvider to NJ22. This will increase
our knowledge of NetCDF also. The end result is an added package for NJ22 -
This might require me having access to your CVS (if you prefer me committing
this to your package - or I will have to have it on our side).
In phase-2, I could read about the common data model and try to see how it
could be incorporated for SRB objects. I really don't know much here and
need to read more or get a knowledge dump from anyone from your team working
on CDM. We already support some small HDF function calls in SRB. The idea
here is that the user could access NetCDF (or CDM files) that are
distributed in multiple sites as though they are local. SRB would manage
global namespace of all the distributed files, search based on meta data,
perform data archival and other data management functions. NetCDF could
provide the formatting or syntactic layer required to access data. We have
done some thing similar for HDF5 before for a subset of HDF functions. We
could do something similar for NetCDF or CDM (which ever would be
appropriate).
Let me know if you have any suggestions on how I could proceed. I would also
require CVS access (at a lager stage) to NJ22 if you prefer me committing
the SRB IOProvider as another package.
Thanks,
Arun
~~~~~~~~~
Luck is what happens when preparation meets opportunity.
Arun Jagatheesan
http://www.sdsc.edu/~arun/
San Diego Supercomputer Center.
(858)822.5452
