Re: [thredds] TDM 4.5 fail to trigger TDS rescan

  • To: "John Caron" <caron@xxxxxxxx>
  • Subject: Re: [thredds] TDM 4.5 fail to trigger TDS rescan
  • From: 唐健 <tangjian@xxxxxxxxxx>
  • Date: Wed, 31 Dec 2014 14:14:03 +0800 (GMT+08:00)
Thanks John!


After I enabled SSL on tomcat, everything now is working fine!!
the miss spelling has also been corrected, can't believe I did that...lol

Thank you very much!
Happy New Year!

Best,
Jian


-----原始邮件-----
发件人:"John Caron" <caron@xxxxxxxx>
发送时间:2014-12-31 10:45:39 (星期三)
收件人: "唐健" <tangjian@xxxxxxxxxx>
抄送: "THREDDS community" <thredds@xxxxxxxxxxxxxxxx>
主题: Re: [thredds] TDM 4.5 fail to trigger TDS rescan


Hi Jian:


Do you have SSL enabled on your tomcat server?
Can you manually hit those URLs successfully?
I see you have "tdm:sercret" but password="secret" 


see: 
http://www.unidata.ucar.edu/software/thredds/v4.5/tds/reference/collections/FeatureCollections.html#trigger


John


On Fri, Dec 26, 2014 at 1:34 AM, 唐健 <tangjian@xxxxxxxxxx> wrote:
Hi there,  Merry Christmas to you all

I have been upgrading our TDS from 4.3 to Version 4.5.4

My tomcat version is 7.0.47
Java (jdk1.7.0_55)
the whole system runing on Windows Server 2008 R2 Standard(X86_64),which has a 
RAM of 24GB

The upgrading run smoothly but one thing, when the grib-datasets is change, the 
tdm fail to trigger tds

I stand by the
http://www.unidata.ucar.edu/software/thredds/v4.5/tds/reference/collections/TDM.html
tell me how to start TDM, but still I think I am doing something wrong here.

Please, if anyone knows whats going on, please point out for me
Thank you very much!

one of the catalog looks like this for example

============================================================================================================================================================================
<?xml version="1.0" encoding="UTF-8"?>
<catalog name="NMC THREDDS Server Catalog" 
xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
 xmlns:xlink="http://www.w3.org/1999/xlink";  version="1.0.2" >

<service name="Default" serviceType="Compound" base="">
  <service name="opendap" serviceType="OPENDAP" base="/thredds/dodsC/"/>
  <service name="fileServer" serviceType="HTTPServer" 
base="/thredds/fileServer/"/>
</service>

<featureCollection name="ECMWF_Main_High_Resolution_Model_Products_(C1D)" 
featureType="GRIB1" path="nwp/ECMWF/C1D" >
  <metadata inherited="true">
     <serviceName>Default</serviceName>
     <dataType>GRID</dataType>
     <documentation type="summary">Single reference time Grib 
Collection</documentation>
     <documentation type="summary">
       ECMWF Main High Resolution Model Products (C1D)
     </documentation>
  </metadata>
  <collection spec="D:/THREDDS_NEW/nwp/ECMWF/C1D/.*grib1$"
              timePartition="file"
              dateFormatMark="yyyyMMddHH" />
  <gribConfig datasetTypes="TwoD Latest Best" />
  <update startup="never" trigger="allow"/>
  <tdm rewrite="test" rescan="0 0/15 * * * ? *" trigger="allow"/>
</featureCollection>

</catalog>
============================================================================================================================================================================
the log looks like this
============================================================================================================================================================================
2014-12-26T14:41:26.094 +0800 INFO  - FeatureCollection config=name 
='ECMWF_Main_High_Resolution_Model_Products_(C1D)' type='GRIB1'
  spec='D:/THREDDS_NEW/nwp/ECMWF/C1D/.*grib1$'
  dateFormatMark ='yyyyMMddHH'
  timePartition =file
  updateConfig =UpdateConfig{userDefined=true, recheckAfter='null', 
rescan='null', triggerOk=true, updateType=never}
  tdmConfig =UpdateConfig{userDefined=true, recheckAfter='null', rescan='0 0/15 
* * * ? *', triggerOk=true, updateType=test}
  ProtoConfig{choice=Penultimate, change='null', param='null', 
outerNcml='null', cacheAll=true}
  hasInnerNcml =false
  GribConfig{datasets=[TwoD, Best, Latest], pdsHash={useGenType=false, 
intvMerge=true, useTableVersion=true}, filesSortIncreasing=true}

2014-12-26T14:41:26.094 +0800 INFO  - Schedule startup scan force=test for 
'ECMWF_Main_High_Resolution_Model_Products_(C1D)' at 
2014-12-26T14:41:29.094+0800
2014-12-26T14:41:26.094 +0800 INFO  - Schedule recurring scan for 
'ECMWF_Main_High_Resolution_Model_Products_(C1D)' cronExpr=0 0/15 * * * ? *
2014-12-26T14:41:29.120 +0800 INFO  - updateGribCollection 
ECMWF_Main_High_Resolution_Model_Products_(C1D) changed false took 16 msecs
2014-12-26T14:45:00.095 +0800 INFO  - updateGribCollection 
ECMWF_Main_High_Resolution_Model_Products_(C1D) changed false took 16 msecs
2014-12-26T15:00:00.045 +0800 INFO  - updateGribCollection 
ECMWF_Main_High_Resolution_Model_Products_(C1D) changed false took 16 msecs
2014-12-26T15:15:00.042 +0800 INFO  - updateGribCollection 
ECMWF_Main_High_Resolution_Model_Products_(C1D) changed false took 16 msecs
2014-12-26T15:30:00.070 +0800 INFO  - updateGribCollection 
ECMWF_Main_High_Resolution_Model_Products_(C1D) changed false took 16 msecs
2014-12-26T15:45:00.035 +0800 INFO  - updateGribCollection 
ECMWF_Main_High_Resolution_Model_Products_(C1D) changed false took 0 msecs
2014-12-26T15:48:32.071 +0800 INFO  - FeatureCollection config=name 
='ECMWF_Main_High_Resolution_Model_Products_(C1D)' type='GRIB1'
  spec='D:/THREDDS_NEW/nwp/ECMWF/C1D/.*grib1$'
  dateFormatMark ='yyyyMMddHH'
  timePartition =file
  updateConfig =UpdateConfig{userDefined=true, recheckAfter='null', 
rescan='null', triggerOk=true, updateType=never}
  tdmConfig =UpdateConfig{userDefined=true, recheckAfter='null', rescan='0 0/15 
* * * ? *', triggerOk=true, updateType=test}
  ProtoConfig{choice=Penultimate, change='null', param='null', 
outerNcml='null', cacheAll=true}
  hasInnerNcml =false
  GribConfig{datasets=[TwoD, Best, Latest], pdsHash={useGenType=false, 
intvMerge=true, useTableVersion=true}, filesSortIncreasing=true}

2014-12-26T15:48:32.071 +0800 INFO  - Schedule startup scan force=test for 
'ECMWF_Main_High_Resolution_Model_Products_(C1D)' at 
2014-12-26T15:48:35.071+0800
2014-12-26T15:48:32.071 +0800 INFO  - Schedule recurring scan for 
'ECMWF_Main_High_Resolution_Model_Products_(C1D)' cronExpr=0 0/15 * * * ? *
2014-12-26T15:48:46.095 +0800 INFO  - GribCollectionBuilder write 
D:/THREDDS_NEW/nwp/ECMWF/C1D/2014122512.grib1.ncx2 ok=true
2014-12-26T15:48:47.031 +0800 ERROR - gc2tp cant delete 
D:\THREDDS_NEW\nwp\ECMWF\C1D\ECMWF_Main_High_Resolution_Model_Products_(C1D)-C1D.ncx2
2014-12-26T15:48:47.078 +0800 INFO  - RewriteFilePartition 
ECMWF_Main_High_Resolution_Model_Products_(C1D)-C1D took 11997 msecs
2014-12-26T15:48:47.078 +0800 INFO  - updateGribCollection 
ECMWF_Main_High_Resolution_Model_Products_(C1D) changed true took 11997 msecs
2014-12-26T15:58:47.476 +0800 ERROR - FAIL send trigger to 
http://thredds.cma.gov.cn/thredds/admin/collection/trigger?trigger=never&collection=ECMWF_Main_High_Resolution_Model_Products_(C1D)
 failed
java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.7.0_55]
        at java.net.SocketInputStream.read(SocketInputStream.java:152) 
~[?:1.7.0_55]
        at java.net.SocketInputStream.read(SocketInputStream.java:122) 
~[?:1.7.0_55]
        at sun.security.ssl.InputRecord.readFully(InputRecord.java:442) 
~[?:1.7.0_55]
        at sun.security.ssl.InputRecord.read(InputRecord.java:480) ~[?:1.7.0_55]
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927) 
~[?:1.7.0_55]
        at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) 
~[?:1.7.0_55]
        at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:702) 
~[?:1.7.0_55]
        at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122) 
~[?:1.7.0_55]
        at 
org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:147)
 ~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at 
org.apache.http.impl.io.AbstractSessionOutputBuffer.flush(AbstractSessionOutputBuffer.java:154)
 ~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at 
org.apache.http.impl.AbstractHttpClientConnection.doFlush(AbstractHttpClientConnection.java:278)
 ~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at 
org.apache.http.impl.AbstractHttpClientConnection.flush(AbstractHttpClientConnection.java:283)
 ~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at 
org.apache.http.impl.conn.ManagedClientConnectionImpl.flush(ManagedClientConnectionImpl.java:175)
 ~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at 
org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:260)
 ~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at 
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
 ~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at 
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
 ~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at 
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
 ~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at 
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
 ~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at 
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
 ~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at ucar.httpservices.HTTPSession.execute(HTTPSession.java:898) 
~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at ucar.httpservices.HTTPMethod.execute(HTTPMethod.java:306) 
~[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at thredds.tdm.Tdm$IndexTask.sendTriggers(Tdm.java:360) 
[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at thredds.tdm.Tdm$IndexTask.run(Tdm.java:331) 
[tdm-4.5.jar:4.5.4-SNAPSHOT]
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[?:1.7.0_55]
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[?:1.7.0_55]
        at java.lang.Thread.run(Thread.java:745) [?:1.7.0_55]
2014-12-26T16:00:00.079 +0800 INFO  - updateGribCollection 
ECMWF_Main_High_Resolution_Model_Products_(C1D) changed false took 16 msecs
2014-12-26T16:25:04.686 +0800 INFO  - updateGribCollection 
ECMWF_Main_High_Resolution_Model_Products_(C1D) changed false took 16 msecs

============================================================================================================================================================================
The script I use to start TDM is
"C:/Program Files/Java/jdk1.7.0_55/bin/java" -Xmx4g 
-Dtds.content.root.path="C:\Program Files\Apache Software Foundation\Tomcat 
7.0\content" -jar tdm-4.5.jar -tds "http://thredds.cma.gov.cn/"; -cred 
tdm:sercret
and I already add this to tomcat-users.xml
  <role rolename="tdsTrigger"/>
  <user username="tdm" password="secret" roles="tdsTrigger"/>

============================================================================================================================================================================


--
Dr. Jian TANG
Central Meteorological Office
National Meteorological Center of CMA

No. 46, Zhongguancun South Street, Haidian District, Beijing
P. R. China, 100081
Tel: +86 0 13717637500
      +86 10 68400450
Email: tangjian@xxxxxxxxxx


_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/





--

Dr. Jian TANG
Central Meteorological Office
National Meteorological Center of CMA

No. 46, Zhongguancun South Street, Haidian District, Beijing
P. R. China, 100081

  • 2014 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: