Doug Lindholm wrote:
John Caron wrote:
I think last time we discussed this, I wrote:
"The problem is that a location doesnt specify the IOSP needed to handle
it. We have to ask all the IOSPs we know about to find out who handles
it. The IOSP needs to examine the contents of the dataset to decide if
"its mine". Rather than having each IOSP open the file, we pass each the
RandomAccessFile, for efficiency. The IOSP design is obviously oriented
to using RandomAccessFile.
I figured that if an iosp is defined in the NcML, then there is no need
to go door-to-door with an open file, so NetCDF could ignore the
"location" and let the IOSP deal with it. However, that would mean
different behavior for registered IOSPs and custom IOSPs.
"Now an NcML file can specify the IOSP, so potentially this might be a
workaround this problem. The presumed use of NcML is to wrap an existing
NetcdfFile. You want to reverse that and let your IOSP's NetcdfFile wrap
the NcML, by giving it access to the NcML.
I'm taking my inspiration from things like the GRIB IOSP. My vision of
NcML is that it exposes data virtually, in terms of the Common Data
Model, enabling the NetCDF API to read it. That is much more powerful
than simply enhancing an existing NetCDF file.
My dilemma has been one of metadata. Many of my data sources don't have
sufficient information to define a NetcdfFile. I needed a way to provide
metadata and NcML seemed like the ideal format. In the interest of not
repeating myself, my NcML serves that dual role. Otherwise, I could put
the redundant metadata in a different file that I would have to identify
in the NcML. Then I'd have maintenance issues.
"I could see trying an experimental version that passes the NcML to the
iosp specified in the NcML. I would guess that NcMLReader would not
further wrap it, though maybe we can relax that later. Im not sure what
issues we'd find, and but we could try it, though i wouldnt commit to
supporting it until we have some experience.
Initially, I self-identified the NcML file location in the iospParams.
That just felt wrong and meant that I had to edit the file if I wanted
to relocate it. I then modified the NetCDF code to give me access to the
NcML location from the NcMLReader$NcMLNetcdfFile that is handed to my
IOSP's "open". After all, it is a "NcMLNetcdfFile". There were
complications with aggregations, so now I provide access to the JDOM
"netcdf" Element. It is quite harmless in itself. Worst case, I do
recall that JDOM Lists are live so you might be giving the IOSP
developer a way to shoot himself in the foot. But if they are modifying
those, then they deserve it. :-)
"The question is whether an IOSP is the right level to implement your
database access. Alternatively, one could subclass NetcdfFile, which is
what other non-file implementations like opendap, thredds, and cdmremote
do. The problem is that theres no mechanism to associate the location
with a factory that creates those subclasses. I suppose we could create
one, though.
"So have you considered subclassing NetcdfFile instead of writing an
IOSP ?
Again, I'm modeling my use of the IOSP after GRIB and the others. I want
a data provider to have a simple way to server data. Defining the data
structure in terms of the CDM with NcML seems ideal and very powerful.
Im not sure what you get from the GRIB IOSP, it uses an RAF in the usual way.
Giving an IOSP access to the NcML JDOM is probably ok (im not totally sure). Im
not sure how much of the NcMLReader machinery you can take advantage of. You do
have to be careful of pushing a technolgy into places it wasnt designed for. I
should probably look at you NcMLReader$NcMLNetcdfFile mods to see if they can
be added to the base.
