[thredds] TDS 4.6 - I still can't get GRIB aggregations to work

Hi All:

I have a bunch of FNMOC Grib1 pressure field.  My aggregations work fine in TDS 
4.3.x, but I have not been able to get them to work in either 4.5 or now 4.6.  
I am probably being brain-dead or missing something obvious, but here is the 
error message I am getting:


> 2015-04-09T08:29:13.343 -0700 INFO  - Schedule startup scan force=always for 
> 'FNMOC_NAVGEM_0.5-degree_6-hourly_Pressure' at 2015-04-09T08:29:16.339-0700
> 2015-04-09T08:29:16.817 -0700 ERROR - UpdateCollectionJob.execute startup 
> failed collection=FNMOC_NAVGEM_0.5-degree_6-hourly_Pressure
> java.lang.RuntimeException: SimpleDateFormat = yyyyMMddHH fails on 
> FNMOC_NAVGEM_0.5-degree_6-hourly_Pressure-201302.ncx3 
> ParseException:Unparseable date: "ssure-2013"
>       at 
> ucar.nc2.units.DateFromString.getDateUsingCompleteDateFormatWithOffset(DateFromString.java:253)
>  ~[cdm-4.6.0.jar:4.6.0]
>       at 
> ucar.nc2.units.DateFromString.getDateUsingDemarkatedCount(DateFromString.java:99)
>  ~[cdm-4.6.0.jar:4.6.0]
>       at 
> thredds.inventory.DateExtractorFromName.getCalendarDateFromPath(DateExtractorFromName.java:86)
>  ~[cdm-4.6.0.jar:4.6.0]
>       at 
> ucar.nc2.grib.collection.PartitionCollectionMutable$Partition.<init>(PartitionCollectionMutable.java:364)
>  ~[grib-4.6.0.jar:4.6.0]
>       at 
> ucar.nc2.grib.collection.PartitionCollectionMutable.addPartition(PartitionCollectionMutable.java:411)
>  ~[grib-4.6.0.jar:4.6.0]
>       at 
> ucar.nc2.grib.collection.GribPartitionBuilder.createPartitionedIndex(GribPartitionBuilder.java:139)
>  ~[grib-4.6.0.jar:4.6.0]
>       at 
> ucar.nc2.grib.collection.GribCdmIndex.updatePartition(GribCdmIndex.java:376) 
> ~[grib-4.6.0.jar:4.6.0]
>       at 
> ucar.nc2.grib.collection.GribCdmIndex.updateDirectoryCollectionRecurse(GribCdmIndex.java:465)
>  ~[grib-4.6.0.jar:4.6.0]
>       at 
> ucar.nc2.grib.collection.GribCdmIndex.updateGribCollection(GribCdmIndex.java:341)
>  ~[grib-4.6.0.jar:4.6.0]
>       at 
> ucar.nc2.grib.collection.GribCdmIndex.openGribCollection(GribCdmIndex.java:673)
>  ~[grib-4.6.0.jar:4.6.0]
>       at 
> thredds.catalog.InvDatasetFcGrib.updateCollection(InvDatasetFcGrib.java:151) 
> ~[tdcommon-4.6.0.jar:4.6.0]
>       at 
> thredds.catalog.InvDatasetFeatureCollection.checkState(InvDatasetFeatureCollection.java:309)
>  ~[tdcommon-4.6.0.jar:4.6.0]
>       at 
> thredds.catalog.InvDatasetFeatureCollection.update(InvDatasetFeatureCollection.java:330)
>  ~[tdcommon-4.6.0.jar:4.6.0]
>       at 
> thredds.catalog.InvDatasetFeatureCollection.sendEvent(InvDatasetFeatureCollection.java:226)
>  ~[tdcommon-4.6.0.jar:4.6.0]
>       at 
> thredds.inventory.CollectionUpdater$UpdateCollectionJob.execute(CollectionUpdater.java:242)
>  [cdm-4.6.0.jar:4.6.0]
>       at org.quartz.core.JobRunShell.run(JobRunShell.java:207) 
> [quartz-2.2.0.jar:?]
>       at 
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:560) 
> [quartz-2.2.0.jar:?]
> 2015-04-09T08:34:00.754 -0700 INFO  - FeatureCollection added = 
> FeatureCollectionConfig name ='FNMOC NAVGEM 0.5-degree 6-hourly Pressure' 
> collectionName='FNMOC_NAVGEM_0.5-degree_6-hourly_Pressure' type='GRIB1'
>   
> spec='/u00/FNMOC/NAVGEM/pressure/**/US058GMET-GR1mdl.0018_0056_00000F0..#yyyyMMddHH#_0102_000000-000000pres$'
>   olderThan =5 min
>   timePartition =directory
>   updateConfig =UpdateConfig{userDefined=true, recheckAfter='null', 
> rescan='null', triggerOk=true, updateType=always}
>   tdmConfig =UpdateConfig{userDefined=true, recheckAfter='null', 
> rescan='null', triggerOk=true, updateType=always}
>   ProtoConfig{choice=Penultimate, change='null', param='null', 
> outerNcml='null', cacheAll=true}
>   hasInnerNcml =false
>   GribConfig{datasets=[Best, Latest], useGenType=false, 
> useTableVersion=false, intvMerge=true, useCenter=false, 
> filesSortIncreasing=true}


Here is the relevant part of  the catalog:

>      <featureCollection name="FNMOC NAVGEM 0.5-degree 6-hourly Pressure" 
> featureType="GRIB1" harvest="true" path="FNMOC/navgem/pressure/6hr">
>        <metadata inherited="true">
>          <dataType>GRID</dataType>
>           <dataFormat>GRIB-1</dataFormat>
>           <serviceName>all</serviceName>
>           <documentation type="summary">
>               FNMOC 1-degree pressure fields from the NOGAPS model
>            </documentation>
>        </metadata>
> 
>        <collection 
> spec="/u00/FNMOC/NAVGEM/pressure/**/US058GMET-GR1mdl.0018_0056_00000F0..#yyyyMMddHH#_0102_000000-000000pres$"
>            timePartition="directory"    olderThan="5 min"/>
> 



This is my test directory structure:

> root@localhost pressure]# pwd
> /u00/FNMOC/NAVGEM/pressure
> [root@localhost pressure]# ls
> 201302
> 


Here are typical files  (note some have F0FL and some F0OF which why the "F0..” 
in the time string

> US058GMET-GR1mdl.0018_0056_00000F0RL2013021400_0102_000000-000000pres
> US058GMET-GR1mdl.0018_0056_00000F0OF2013021306_0102_000000-000000pres


Some things on the internet suggest that files have to have “.grib1” at the 
end,  Is this true?  If not, if anyone can see what my error is I sure would 
appreciate it.  This is why I haven’t upgraded.

Thanks,

-Roy

**********************
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new address and phone***
110 Shaffer Road
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: Roy.Mendelssohn@xxxxxxxx www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.



  • 2015 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: