Re: [thredds] Add non time dependent variable to FMRC

Hi,


On 03/04/18 14:51, marceloandrioni@xxxxxxxxxxxxxxxx wrote:
Hello, I have a FMRC aggregation of several yearly files with the usual 
variables (ssh,t,s,u,v), e.g.

<featureCollection name="mercator" featureType="FMRC" harvest="true" 
path="mercator">
   <collection spec="/home/opendap/datasets/mercator/**/mercator_#yyyy#\.nc"/>
   <update startup="true"/>
   <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
   </netcdf>
   <fmrcConfig regularize="false" datasetTypes="Best Files" />
</featureCollection>

I have a different file with non time dependent variables (bathymetry, sea land mask, etc) and 
would like to have these variables also appear in the FMRC. I tried just renaming the file to 
mercator_1970.nc hoping that the <collection 
spec="/home/opendap/datasets/mercator/**/mercator_#yyyy#\.nc"/> would catch it, 
but it didn't work. My guess is that doesn't work because there is no time dimension in this 
file and time is necessary for FMRC aggregation.
No. It's because the prototype dataset been chosen by the featureCollection doesn't contain those variables. By default the featureCollection choose the penultimate dataset/file to build-up the resulting dataset with all dataset/files in the collection.
Besides appending the non time dependent variables to every single file with
ncks -A bathymetry_mask.nc mercator_YYYY.nc
is there a way using ncml to join all the files (with and without time 
dimensions) in the FMRC?
You can change the default prototype dataset by:

<featureCollection name="mercator" featureType="FMRC" harvest="true" 
path="mercator">
  <collection spec="/home/opendap/datasets/mercator/**/mercator_#yyyy#\.nc"/>
  <update startup="true"/>
*<protoDataset **choice**="First"/>*
  <fmrcConfig regularize="false" datasetTypes="Best Files" />
</featureCollection>

and append the missing variables to the first dataset/file in the collection, 
i.e.
ncks -A bathymetry_mask.nc mercator_1970.nc

Then they should be "appear" in the resulting dataset. From the doc [1] you can 
read
/.... The choice of the protoDataset matters when the datasets are not homogenous://////Global and variable attributes are taken from the prototype dataset.////If a variable appears in the prototype dataset, it will appear in the feature collection dataset. If it doesnt appear in other datasets, it will have missing data for those times.////If a variable does not appears in the prototype dataset, it will not appear in the feature collection dataset, even if it appears in other datasets./
....


Also, I am adamant in using FMRC because my tests showed that is a lot faster 
to retrieve data than using aggregation joinNew or joinExisting.
In my case I'm reluctant to use FMRC because you have less control on building the resulting dataset. With pure ncml and aggregations you have better control on resulting dataset and good performance if you control cache and definition of the coordinate values.

Regards

Antonio

[1] https://www.unidata.ucar.edu/software/thredds/v4.6/tds/reference/collections/FeatureCollections.html#elements



--
Antonio S. Cofiño
Dep. de Matemática Aplicada y
        Ciencias de la Computación
Universidad de Cantabria
http://www.meteo.unican.es

  • 2018 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: