Hi Roy:
java.lang.RuntimeException: SimpleDateFormat = yyyyMMddHH fails on
FNMOC_NAVGEM_0.5-degree_6-hourly_Pressure-201302.ncx3
ParseException:Unparseable date: "ssure-2013"
So it looks like the index files (ncx3) are in the data directory, so those
need to get excluded. If all of your files end in "pres", you could use:
<collection spec="/u00/FNMOC/NAVGEM/pressure/**/.*pres$"
timePartition="directory"
olderThan="5 min"/>
The problem is that you need to exclude the non GRIB files fron your
collection, so change your collection element to:
<collection spec="/u00/FNMOC/NAVGEM/pressure/**/.*grib1$"
timePartition="directory"
olderThan="5 min"/>
Note that GRIB collections no longer need the date filename parser (unless
you are using timePartition=<time duration>) so we dont need the
<collection spec="/u00/FNMOC/NAVGEM/pressure/**/US058GMET-GR1mdl.
0018_0056_00000F0..#yyyyMMddHH#_0102_000000-000000pres$"
> timePartition="directory" olderThan="5 min"/>
On Thu, Apr 9, 2015 at 12:34 PM, Roy Mendelssohn - NOAA Federal <
roy.mendelssohn@xxxxxxxx> wrote:
> Hi All:
>
> I have a bunch of FNMOC Grib1 pressure field. My aggregations work fine
> in TDS 4.3.x, but I have not been able to get them to work in either 4.5 or
> now 4.6. I am probably being brain-dead or missing something obvious, but
> here is the error message I am getting:
>
>
> > 2015-04-09T08:29:13.343 -0700 INFO - Schedule startup scan force=always
> for 'FNMOC_NAVGEM_0.5-degree_6-hourly_Pressure' at
> 2015-04-09T08:29:16.339-0700
> > 2015-04-09T08:29:16.817 -0700 ERROR - UpdateCollectionJob.execute
> startup failed collection=FNMOC_NAVGEM_0.5-degree_6-hourly_Pressure
> > java.lang.RuntimeException: SimpleDateFormat = yyyyMMddHH fails on
> FNMOC_NAVGEM_0.5-degree_6-hourly_Pressure-201302.ncx3
> ParseException:Unparseable date: "ssure-2013"
> > at
> ucar.nc2.units.DateFromString.getDateUsingCompleteDateFormatWithOffset(DateFromString.java:253)
> ~[cdm-4.6.0.jar:4.6.0]
> > at
> ucar.nc2.units.DateFromString.getDateUsingDemarkatedCount(DateFromString.java:99)
> ~[cdm-4.6.0.jar:4.6.0]
> > at
> thredds.inventory.DateExtractorFromName.getCalendarDateFromPath(DateExtractorFromName.java:86)
> ~[cdm-4.6.0.jar:4.6.0]
> > at
> ucar.nc2.grib.collection.PartitionCollectionMutable$Partition.<init>(PartitionCollectionMutable.java:364)
> ~[grib-4.6.0.jar:4.6.0]
> > at
> ucar.nc2.grib.collection.PartitionCollectionMutable.addPartition(PartitionCollectionMutable.java:411)
> ~[grib-4.6.0.jar:4.6.0]
> > at
> ucar.nc2.grib.collection.GribPartitionBuilder.createPartitionedIndex(GribPartitionBuilder.java:139)
> ~[grib-4.6.0.jar:4.6.0]
> > at
> ucar.nc2.grib.collection.GribCdmIndex.updatePartition(GribCdmIndex.java:376)
> ~[grib-4.6.0.jar:4.6.0]
> > at
> ucar.nc2.grib.collection.GribCdmIndex.updateDirectoryCollectionRecurse(GribCdmIndex.java:465)
> ~[grib-4.6.0.jar:4.6.0]
> > at
> ucar.nc2.grib.collection.GribCdmIndex.updateGribCollection(GribCdmIndex.java:341)
> ~[grib-4.6.0.jar:4.6.0]
> > at
> ucar.nc2.grib.collection.GribCdmIndex.openGribCollection(GribCdmIndex.java:673)
> ~[grib-4.6.0.jar:4.6.0]
> > at
> thredds.catalog.InvDatasetFcGrib.updateCollection(InvDatasetFcGrib.java:151)
> ~[tdcommon-4.6.0.jar:4.6.0]
> > at
> thredds.catalog.InvDatasetFeatureCollection.checkState(InvDatasetFeatureCollection.java:309)
> ~[tdcommon-4.6.0.jar:4.6.0]
> > at
> thredds.catalog.InvDatasetFeatureCollection.update(InvDatasetFeatureCollection.java:330)
> ~[tdcommon-4.6.0.jar:4.6.0]
> > at
> thredds.catalog.InvDatasetFeatureCollection.sendEvent(InvDatasetFeatureCollection.java:226)
> ~[tdcommon-4.6.0.jar:4.6.0]
> > at
> thredds.inventory.CollectionUpdater$UpdateCollectionJob.execute(CollectionUpdater.java:242)
> [cdm-4.6.0.jar:4.6.0]
> > at org.quartz.core.JobRunShell.run(JobRunShell.java:207)
> [quartz-2.2.0.jar:?]
> > at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:560)
> [quartz-2.2.0.jar:?]
> > 2015-04-09T08:34:00.754 -0700 INFO - FeatureCollection added =
> FeatureCollectionConfig name ='FNMOC NAVGEM 0.5-degree 6-hourly Pressure'
> collectionName='FNMOC_NAVGEM_0.5-degree_6-hourly_Pressure' type='GRIB1'
> >
>
> spec='/u00/FNMOC/NAVGEM/pressure/**/US058GMET-GR1mdl.0018_0056_00000F0..#yyyyMMddHH#_0102_000000-000000pres$'
> > olderThan =5 min
> > timePartition =directory
> > updateConfig =UpdateConfig{userDefined=true, recheckAfter='null',
> rescan='null', triggerOk=true, updateType=always}
> > tdmConfig =UpdateConfig{userDefined=true, recheckAfter='null',
> rescan='null', triggerOk=true, updateType=always}
> > ProtoConfig{choice=Penultimate, change='null', param='null',
> outerNcml='null', cacheAll=true}
> > hasInnerNcml =false
> > GribConfig{datasets=[Best, Latest], useGenType=false,
> useTableVersion=false, intvMerge=true, useCenter=false,
> filesSortIncreasing=true}
>
>
> Here is the relevant part of the catalog:
>
> > <featureCollection name="FNMOC NAVGEM 0.5-degree 6-hourly Pressure"
> featureType="GRIB1" harvest="true" path="FNMOC/navgem/pressure/6hr">
> > <metadata inherited="true">
> > <dataType>GRID</dataType>
> > <dataFormat>GRIB-1</dataFormat>
> > <serviceName>all</serviceName>
> > <documentation type="summary">
> > FNMOC 1-degree pressure fields from the NOGAPS model
> > </documentation>
> > </metadata>
> >
> > <collection
> spec="/u00/FNMOC/NAVGEM/pressure/**/US058GMET-GR1mdl.0018_0056_00000F0..#yyyyMMddHH#_0102_000000-000000pres$"
> > timePartition="directory" olderThan="5 min"/>
> >
>
>
>
> This is my test directory structure:
>
> > root@localhost pressure]# pwd
> > /u00/FNMOC/NAVGEM/pressure
> > [root@localhost pressure]# ls
> > 201302
> >
>
>
> Here are typical files (note some have F0FL and some F0OF which why the
> "F0..” in the time string
>
> > US058GMET-GR1mdl.0018_0056_00000F0RL2013021400_0102_000000-000000pres
> > US058GMET-GR1mdl.0018_0056_00000F0OF2013021306_0102_000000-000000pres
>
>
> Some things on the internet suggest that files have to have “.grib1” at
> the end, Is this true? If not, if anyone can see what my error is I sure
> would appreciate it. This is why I haven’t upgraded.
>
> Thanks,
>
> -Roy
>
> **********************
> "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> ***Note new address and phone***
> 110 Shaffer Road
> Santa Cruz, CA 95060
> Phone: (831)-420-3666
> Fax: (831) 420-3980
> e-mail: Roy.Mendelssohn@xxxxxxxx www: http://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward justice" -MLK
> Jr.
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
