[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Revised: StationProfileFeatureCollection does not implement getPointFeatureCollec,tionIterator()



Hi Bob:

If you go to ToolsUI/FeatureTypes/FeatureScan and open a directory with these files in it, it will try to open each file and tell you what it finds.

I find that StationProfile and the epa file fails, but the other 4 succeed. If you select a file and right-click, you get a context menu from which you can jump to other tabs. jump to "open as point dataset" and click around to explore the data. this is what the standard PointFeature factory does with the files.

My guess is that theres a bug in the flatten() method, which the above doesnt use. Ill try to get a unit test on those files to see.

You are seeing the immaturity of the design and the implementation. I guess well just take things one at a time and try to fix the problems.

a few more comments below (more later)


On 2/3/2011 4:20 PM, Bob Simons wrote:
Attached are 5 .nc point files created by me (all use the various "ragged array (contiguous) representations") and 1 file (epa) created by the SURA group. All are intended to follow the proposed CF Point Observations conventions at
https://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventions .
But since this is new to us, we may have made mistakes -- that could certainly be the source of the problems below. But I see nothing wrong with the 5 files created by me. Basically, I couldn't read data from any file except a Point file (not attached).
I'm using a very recent version of netcdf-java 4.2.

(Some) Problems
* For Profile.nc, Station.nc, Trajectory.nc, (and TrajectoryProfile.nc and epa+seamap_04-08_WD23.nc are analogous), I see the same problem:
   I use flatten() to get one PointFeatureCollection
(or the iterator to get each e.g., ProfileFeature, which is a PointFeatureCollection))
   then I use the internal iterator to get the PointFeatures
   and then pointFeature.getData().getMembers to get the members
   then getArray to get the data.
   But the size of each array varies.
      For longitude, latitude, time, rowSize, (station), the size
         is the number of (stations).
      For all the actual data variables, the size is one.
      I would have expected the sizes to all be 1.

* For TrajectoryProfile.nc (for a SectionFeatureCollection)
  my code uses layers of iterators, but is roughly similar to
  e.g, Profile.nc described above.
  * The first problem described above (different sized arrays)
    occurs here, too.
  * There are no members for the actual data variables.
  * member=bottle_posn getDataArray() always returns null!
    (For one of the other files, member=altitude getDataArray()
    always returns null.)

* The problems with epa+seamap_04-08_WD23.nc are similar to the problems with TrajectoryProfile.nc file.

* I never got flatten or subset to work with a dateRange
(even with a huge range, so it should catch all data).
Instead, using dateRange returns a null PointFeatureCollection.
So I always use dateRange=null.

* The JavaDocs say EarthLocation always uses longitude +/-180!
Does that mean that the underlying data always uses longitude +/-180?
Does that mean that when I create files with longitude 0 - 360, that I should convert to +/-180?
What about trajectories that cross -180?

* For the StationProfile.nc file, FeatureDatasetFactoryManager.open() says "Must have a Station id variable with standard name station_id", but I think it does.

this has the station and profile dimensions mixed up. probably a bug.


* For SectionFeatureCollection and StationProfileFeatureCollection,
if I use flatten, then try to use the internal iterator's hasNext() to see the PointFeatures in the PointFeatureCollection, I get Exception in thread "main" java.lang.UnsupportedOperationException: SectionCollectionImpl does not implement getPointFeatureCollectionIterat
or()
at ucar.nc2.ft.point.SectionCollectionImpl.getPointFeatureCollectionIterator(SectionCollectionImpl.java:60) at ucar.nc2.ft.point.MultipleNestedPointCollectionImpl$NestedPointFeatureCollectionFlatten.getPointFeatureIterator(MultipleNestedPoi
ntCollectionImpl.java:108)
at ucar.nc2.ft.point.PointCollectionImpl.resetIteration(PointCollectionImpl.java:89) at ucar.nc2.ft.point.PointCollectionImpl.hasNext(PointCollectionImpl.java:75) at gov.noaa.pfel.erddap.dataset.EDDTableFromNcCFFiles.addDataFromPointFeatureCollection(EDDTableFromNcCFFiles.java:472)
...

* Most of the interfaces don't offer subsetting methods that I would expect, e.g., StationProfileFeatureCollection has no subset(latLonRect).

StationProfileFeatureCollection has a subset(latLonRect) method that is inherited.


* There is little guidance in the JavaDocs:
  * I couldn't find any complete example for how to read (including
    subset) data files. It took a long time to put all the pieces
    together, especially since many of the important classes are
    just interfaces (and so not referenced in other java docs).
  * I open files with FeatureDatasetFactoryManager.open().
    Is that the recommended way?
  * My way of going from FeatureCollection to a specific
    ...Collection interfaces is just to repeated test e.g.,
      if (fc instance of ProfileFeatureCollection) {...}
    then cast to that interface. Is there a better way?
  * Somewhere it says it is imperative that I call
    hasNext()/next() to the end (or finish()).
    Is that true even if an exception if thrown?
    So do I need to use try/catch around everything?
    Won't things be garbage collected when they fall out of scope?
  * What is efficient?
    E.g., when is subset() or flatten() with a latLonRect or
    timeRange efficient vs. not?
  * Does resetIteration() for the internal iterator need to be
    called before initial use of the internal iterators?
    (I saw some sample code that showed this.)

I am probably making mistakes (either when creating the files or reading them). Sorry. But I'm stuck. I can't get anything to work. So any guidance you can provide is appreciated.

Thank you for looking into this.

P.S. If you have other test files for each of the cdm_data_types, can you please send them to me since I only have "ragged array (contiguous) representations" files? I want my code to be able to read all of the possible representations (e.g., multidimensional, ragged array (indexed), single, flattened, ...) of all of the feature types. (E.g., I have no examples of Radial or StationRadial datafiles.) Since flatten() doesn't work reliably, all of the cases don't just resolve to a couple of uses of flatten(). I have to write a little code for each feature type to iterate through various levels down to PointFeatureCollections. It would be great if flatten (with latLonRange and dateRange) worked reliably and efficiently.

Thank you very much.

On 2/1/2011 1:10 PM, John Caron wrote:
Hi Bob:

On 1/31/2011 10:26 AM, Bob Simons wrote:
Please ignore my previous email.

If I use
NestedPointFeatureCollection npfc =
(NestedPointFeatureCollection)stationProfileFeatureCollection;
PointFeatureCollection pfc = npfc.flatten(latLonRect, dateRange);

A call to pfc.resetIteration() throws

Exception in thread "main" java.lang.UnsupportedOperationException:
StationProfileFeatureCollection does not implement getPointFeatureCollec
tionIterator()

Why not?

That seems to be the best (only?) way to specify a latLonRect and
dateRange, and I *want* flattened data.

Im not sure why thats failing, it may be the implementation is not
complete or something deeper. Can you send me a sample file?


Perhaps I'm making a mistake.


Just curious (I have to ask, because one time, for other code, the
answer was 'no'), is all of the Point Feature Collection code finished
and ready to be used?

Im thinking the iteration should work the first time through, ie without
calling resetIteration(), just start calling pfc.hasNext() / next() ??


Sincerely,

Bob Simons
IT Specialist
Environmental Research Division
NOAA Southwest Fisheries Science Center
1352 Lighthouse Ave
Pacific Grove, CA 93950-2079
Phone: (831)658-3205
Fax:   (831)648-8440
Email: address@hidden

The contents of this message are mine personally and
do not necessarily reflect any position of the
Government or the National Oceanic and Atmospheric
Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <><