Re: [thredds] Scan Aggregation vs. Feature Collection

Hi Kyle,

Do you have your FMRC configured to only generate the "Best" dataset? It
sounds, instead, like you are looking at the 2-D time view of the FMRC.

This line in your FMRC would restrict it to only produce the "Best" dataset:

  <fmrcConfig regularize="true" datasetTypes="Best" />

Without an fmrcConfig element, I believe the default is to produce all
views. So the same as

  <fmrcConfig regularize="true" datasetTypes="TwoD Best Files Runs
ConstantForecasts ConstantOffsets" />


Ethan

On 11/28/2012 3:07 PM, Kyle Wilcox wrote:
> I've finally had time to test this out
> 
> New-style featureCollection FMRC: 
> http://64.9.200.113:8080/thredds/catalog/LakeErieSST-FMRC/catalog.html
> Old-style joinExisting Agg: 
> http://64.9.200.113:8080/thredds/aoc.html?dataset=LakeErieSST-Agg
> 
> The TDS catalog entries: https://gist.github.com/4164765
> 
> The NetCDF files are one satellite pass per file.  The time dimension is 
> unlimited with a length of 1 in the individual files.  The time units are 
> seconds since epoch (all even seconds).
> 
> The joinExisting Agg works as expected just like in 4.2.  All of the services 
> (including WMS) work.
> 
> The featureCollection FMRC doesn't work quite as well, but I do get a valid 
> dataset.
>   * There are rounding issues when converting the original files' "seconds 
> since 1970-01-01 00:00:00" to the FMRC's "hours since 
> 2012-03-11T08:00:00.000Z".  Data is no longer on even seconds in the FMRC.  
> Agg: 
> http://64.9.200.113:8080/thredds/dodsC/LakeErieSST-Agg.ascii?time[0:1:132] 
> VS. FMRC: 
> http://64.9.200.113:8080/thredds/dodsC/LakeErieSST-FMRC/LakeErieSST-FMRC_best.ncd.ascii?time[0:1:132]
>   * The datasets has a "time_run" variable even though there are no "runs".  
> I can't remove the "time_run" variable from the resulting dataset using the 
> protoDataset tag.
>   * The "coordinates" attribute of the "sst" variable gets set to "time_run 
> time lon lat".  I don't want 'time_run' to be part of the coordinates.  I 
> can't change the attribute using the protoDataset tag.
>   * The "File_Access" link takes me to a directory listing that doesn't apply 
> the regex of the collection.  It lists all of the files in the directory 
> rather than just the .nc4 files: 
> http://64.9.200.113:8080/thredds/catalog/LakeErieSST-FMRC/files/catalog.html.
>   * The WMS service does not work (it is a known issues with ncWMS and FMRC 
> datasets... it does not handle the run dimension correctly).
> 
> Is a possible solution to create another option in the fmrcConfig's 
> datasetTypes attribute to remove all concept of  a "run" from the dataset?  
> This will be useful for gridded files with no overlapping time.   <fmrcConfig 
> datasetTypes="NoRunsPlzKThnx Files" />
> 
> Another solution could be to remove the concept of a "run" from a FMRC 
> dataset if the calculated time_offset variable is filled with all zeros.
> 
> Thoughts?
> 
> 
> 
>> -----Original Message-----
>> From: John Caron [mailto:caron@xxxxxxxxxxxxxxxx]
>> Sent: Friday, October 26, 2012 7:48 PM
>> To: Kyle Wilcox
>> Cc: Roy Mendelssohn; thredds@xxxxxxxxxxxxxxxx; Charlton Galvarino
>> (2Creek)
>> Subject: Re: [thredds] Scan Aggregation vs. Feature Collection
>>
>> Hi Kyle:
>>
>> For GRID data aggregations, If they are GRIB format, use Feature Collection,
>> type=GRIB. Otherwise use type=FMRC, even if theres only one runtime. Let us
>> know any problems.
>>
>> John
>>
>> On 10/3/2012 10:09 AM, Kyle Wilcox wrote:
>>> Hi John -
>>>
>>> Just trying to clarify a few things when testing 4.3:
>>>
>>> If we have a homogenous set of non-forecast GRID FeatureTypes (satellite),
>> we should still use an 'old-style' NcML aggregation because there is no GRID
>> FeatureCollection.
>>>
>>> If we have a heterogeneous set of non-forecast GRID FeatureTypes (satellite
>> passes on slightly different grids), there is currently no way to aggregate 
>> them
>> in 4.3 because there is no GRID FeatureCollection.
>>>
>>> Thanks ,
>>> Kyle