Doug Lindholm wrote:
I've been developing a few AbstractIOServiceProvider-s (with NetCDF
Java 4.1) to use in my NcML for some custom datasets. I've run into a
few issues (some of which I've already discussed with the developers)
that I'd like to get some insight on.
I have an IOSP that reads data from our relational database. The first
issue I ran into was setting the "location" attribute in my NcML.
There is no file and anything that wasn't a file made netcdf unhappy.
I eventually found that "/dev/null" got me around that problem.
That got me thinking about the possibilities of "location" that
weren't tied to a RandomAccessFile. I'd like to think that a
"location" could be any URI (or URN?) that the IOSP would know how to
deal with.
I then thought that the "location" could be a file that contains the
URI that the IOSP could then read. I didn't like the idea of
maintaining yet another file, so I figured I could use the ncml file
itself. Unfortunately, netcdf is too clever to let me use an ncml file
that refers to itself. Could/should that restriction be relaxed when
an "iosp" is defined?
On a related note, there are often other bits of metadata that I need
for my IOSP when I am serving some data files (e.g. flat binary with
no metadata). Again the ncml file itself is the most practical source
of the metadata my IOSP needs to define variables and such, so I don't
have to keep a redundant set of information. In this case, I am using
the "location" for the data file. The only option I found, beyond a
naming convention, is using the "iospParam" attribute to encode the
location of the ncml so my IOSP can read it. This hack is less than
ideal. I'd like to think that the NetcdfFile that the netcdf code
provides me in the "open" method would have a handle on the ncml file
that defines it. After all, it is an instance of
NcMLReader$NcMLNetcdfFile. Do you see any problem exposing the ncml
location (or even the netcdf Element)? It looks trivial to add an arg
to the NcMLNetcdfFile constructor and a getter to retrieve it.
Any other thoughts or workarounds to give the IOSP more flexibility to
serve datasets that can't easily be represented by single location
that is mapped to a RandomAccessFile?
The problem is that a location doesnt specify the IOSP needed to handle
it. We have to ask all the IOSPs we know about to find out who handles
it. The IOSP needs to examine the contents of the dataset to decide if
"its mine". Rather than having each IOSP open the file, we pass each the
RandomAccessFile, for efficiency. The IOSP design is obviously oriented
to using RandomAccessFile.
Now an NcML file can specify the IOSP, so potentially this might be a
workaround this problem. The presumed use of NcML is to wrap an existing
NetcdfFile. You want to reverse that and let your IOSP's NetcdfFile wrap
the NcML, by giving it access to the NcML.
I could see trying an experimental version that passes the NcML to the
iosp specified in the NcML. I would guess that NcMLReader would not
further wrap it, though maybe we can relax that later. Im not sure what
issues we'd find, and but we could try it, though i wouldnt commit to
supporting it until we have some experience.
The question is whether an IOSP is the right level to implement your
database access. Alternatively, one could subclass NetcdfFile, which is
what other non-file implementations like opendap, thredds, and cdmremote
do. The problem is that theres no mechanism to associate the location
with a factory that creates those subclasses. I suppose we could create
one, though.
So have you considered subclassing NetcdfFile instead of writing an IOSP ?
