I'm struggling to get TDS to update its indices for a realtime GRIB
dataset. I'm running
version= 4.6.14 - 2019-07-23T11:04:31-0600
tdmFat-4.6.14.jar
with the catalog.xml featureCollection set up as:
<featureCollection name="gtgn" featureType="GRIB2" path="gtgn">
<metadata inherited="true">
<serviceName>all</serviceName>
<dataType>Grid</dataType>
</metadata>
<collection name="gtgn"
spec="/d1/data_archive/gtg_ncst_grib/**/gtgn.*\.grb2$"
timePartition="file"
dateFormatMark="gtg_ncst_grib/#yyyyMMdd'/gtgn.t'HHmm#" />
<update startup="never" trigger="allow"/>
<tdm rewrite="test" rescan="0 0/5 * * * ? *" trigger="allow" />
</featureCollection>
When I run the tdm with the following args:
java -Xmx4g -Dtds.content.root.path=/d1/data_archive -jar
/d1/data_archive/thredds/tdmFat-4.6.14.jar -cred "user:password" -log
DEBUG &> /d1/data_archive/tdm/runTdm.log
I see that it correctly regenerates the indicies and triggers:
2020-03-20T13:19:48.644 +0000 DEBUG - createIndex for
/d1/data_archive/gtg_ncst_grib/20200320/gtgn.t0730z.edr.f000.grb2.ncx3
2020-03-20T13:19:48.644 +0000 DEBUG - write RecordMaps: bytes = 675
record = 52 bytesPerRecord=12
2020-03-20T13:19:48.645 +0000 DEBUG - write GribCollectionIndex= 700 bytes
2020-03-20T13:19:48.645 +0000 DEBUG - That took 152 msecs
2020-03-20T13:19:48.658 +0000 DEBUG - Using canonical partition
gtgn.t1300z.edr.f000.grb2
2020-03-20T13:19:48.681 +0000 INFO - RewriteFilePartition gtgn-20200320
took 6648 msecs
2020-03-20T13:19:48.898 +0000 DEBUG - Using canonical partition
gtgn-20200319
2020-03-20T13:19:48.988 +0000 INFO - updateGribCollection gtgn changed
true took 20896 msecs
2020-03-20T13:19:48.989 +0000 DEBUG - 2020-03-20T13:19:48.988Z gtgn changed
true
2020-03-20T13:19:49.013 +0000 INFO - send trigger to
http://localhost:8080/thredds/admin/collection/trigger?trigger=never&collection=gtgn
status = 200
2020-03-20T13:20:01.609 +0000 INFO - updateGribCollection gtgn changed
false took 1269 msecs
2020-03-20T13:20:01.610 +0000 DEBUG - 2020-03-20T13:20:01.609Z gtgn changed
false
However, the catalog (below) never updates:
http://poseidon.rap.ucar.edu:8080/gtgnTDS/catalog/gtgn/catalog.html
even though the WMS getCapabilities (below) DOES update for the new files:
http://poseidon.rap.ucar.edu:8080/thredds/wms/gtgn/TP?service=WMS&version=1.3.0&request=GetCapabilities
I'm relying on the xml catalog to determine run times for some other
forecast datasets, so having updated information is critical to my display
application.
...and a seemingly unrelated oddity is that the getCapabilities' times are
all rounded to the hour, even though the files are 15-minutely. The HTML
catalog gets the times correct (when it's updating correctly, that is), but
the getCapabilities has each 15-minute file listed as being at the top of
the hour (dropping minutes field?). See the latest values in the "time"
dimension below:
<Layer queryable="1">
<Name>Eddy_dissipation_parameter_altitude_above_msl</Name>
<Title>
Eddy dissipation parameter @ Specific altitude above mean sea level
</Title>
<Abstract>
Eddy dissipation parameter @ Specific altitude above mean sea level
</Abstract>
<EX_GeographicBoundingBox>
<westBoundLongitude>-139.97003181300516</westBoundLongitude>
<eastBoundLongitude>-57.26792343693751</eastBoundLongitude>
<southBoundLatitude>16.20865459974452</southBoundLatitude>
<northBoundLatitude>55.51688759604331</northBoundLatitude>
</EX_GeographicBoundingBox>
<BoundingBox CRS="CRS:84" minx="-139.97003181300516" maxx="
-57.26792343693751" miny="16.20865459974452" maxy="55.51688759604331"/>
<Dimension name="elevation" units="m" default="0.0">
0.0,30.0,304.0,609.0,914.0,1219.0,1524.0,1828.0,2133.0,2438.0,2743.0,3048.0,3352.0,3657.0,3962.0,4267.0,4572.0,4876.0,5181.0,5486.0,5791.0,6096.0,6400.0,6705.0,7010.0,7315.0,7620.0,7924.0,8229.0,8534.0,8839.0,9144.0,9448.0,9753.0,10058.0,10363.0,10668.0,10972.0,11277.0,11582.0,11887.0,12192.0,12496.0,12801.0,13106.0,13411.0,13716.0,14020.0,14325.0,14630.0,14935.0,15240.0
</Dimension>
<Dimension name="time" units="ISO8601" multipleValues="true" current="true"
default="2020-03-20T13:00:00.000Z">
2019-11-13T19:00:00.000Z,2019-11-13T20:00:00.000Z,2019-11-13T21:00:00.000Z,2019-11-13T22:00:00.000Z,
...
2020-03-20T11:00:00.000Z,2020-03-20T11:00:00.000Z,2020-03-20T11:00:00.000Z,2020-03-20T11:00:00.000Z,2020-03-20T12:00:00.000Z,2020-03-20T12:00:00.000Z,2020-03-20T12:00:00.000Z,2020-03-20T12:00:00.000Z,2020-03-20T13:00:00.000Z,2020-03-20T13:00:00.000Z,2020-03-20T13:00:00.000Z,2020-03-20T13:00:00.000Z
</Dimension>
Any ideas on what I might be doing wrong or suggestions on how to work
around these issues?
Thanks,
Arnaud
---
Arnaud Dumont
Research Applications Laboratory (RAL)
National Center for Atmospheric Research (NCAR)
3450 Mitchell Ln, Boulder CO, 80301
dumont@xxxxxxxx
+1(303)497-8434
