Hi Jon:
This sounds like a nice idea, which we chatted about briefly at GO-ESSP.
I could see an API like NetcdfDataset.openGlob(glob, type). The tricky part is
to define what it should do in various cases. Im guessing you are handling the
common cases, which is quite nice to make so easy. the optional type is to give
a hint for doing something different, but im not yet sure whats needed.
My first reaction is to roll your code into the ncml package, and have
openGlob() write Ncml and then process it, and then enrich Ncml processing, so
that that functionality is also available in a more explicit way. But id have
to look more at what youve done.
I dont think its an IOSP, but perhaps you have another way of looking at it.
My fmrc implementation seriously needs refactoring, so id like to see what
youve done there also. BTW, how do you identify the time coord vs forecast
time? are you assuming CF conventions?
Jon Blower wrote:
> Dear all (esp. John),
>
> We use NcML a lot for both file aggregation and for "fixing" metadata
> problems in underlying NetCDF files - it's a great technology.
> However, it can be an inconvenience to create an NcML file simply to
> aggregate "well-behaved" files. In our ncWMS we allow users to
> specify a group of files using glob expressions, e.g. "/path/to/*.nc"
> or even more complex things like "/path/to/200?/*/foo.nc". This
> simply unions the matching files together along the time axis. It
> allows files to contain different combinations of variables.
> Internally, the system creates some kind of hash map, so that when a
> user requests a particular variable at a particular time, the
> aggregation knows which actual file, and which time index within the
> file, is being requested.
>
> We have found this to be very useful. I wonder if it would be a good
> idea to integrate this capability into the NetCDF-Java libraries so
> that users can open an aggregation by running
> NetcdfDataset.openDataset("/path/to/*.nc") or similar? What do others
> think?
>
> Our code is available for stealing, but it might need some work to
> satisfy more use cases. In particular, for a forecast model run
> collection (fmrc) our code automatically generates the "best
> timeseries" but doesn't allow access to other things like the run
> dates. I could have a go at creating an IOSP, if this is a good way
> to begin the integration.
>
> Cheers, Jon
>
