Re: [thredds] Authentication problems with the TDS and pydap

  • To: "Pols, Maarten" <M.Pols@xxxxxx>, "thredds@xxxxxxxxxxxxxxxx" <thredds@xxxxxxxxxxxxxxxx>
  • Subject: Re: [thredds] Authentication problems with the TDS and pydap
  • From: Jim Fluke <james.fluke@xxxxxxxxxxxxx>
  • Date: Thu, 11 Jul 2024 16:03:48 -0600
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=colostate.edu; dmarc=pass action=none header.from=colostate.edu; dkim=pass header.d=colostate.edu; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mmx6zafjTM2EyRWt9gGs3kY/dKSdBL19IuXR6pflr6o=; b=WRQxViLVnxoAzpfFKMtMXbb9Dr+f7k0YJ4RkAb8DRI3wUG0ywELf2jjnrkBmfBlh7PnnlEa35shsNK3W4EGSplVB+txwJra43tHGkZFmvfvMBg/JIfJEavq/H6cPO191LLm9YiPDXWNDKu7d8RyvsEFe29fs7MwzPHZQT30Zxbkd3wliGAfCaBHp8WGphvzbxlBC4Wpns4B8SpRyH7UDTmMZj+6cnioCjh2dNI3bJDexmDWRuJiCKmJZA5/WGXa9j1mbH9lyS69YEd2Mw/ACD+IAA7EHCTuzAZ87cabrVMvGZmen8uI/7cRPhyPRNOeo+EWRyoUVAfJcbpozGohmSQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=h4XuDVr9ljhdA+kd84ZVarUL5BIj/zsbkLEhRXdKPR0w+/If/WGczFpgbFPTruBn0DqtVMZClEyWJxTD5Vtueis8nd9AAy/uBFE6QQw5eKMyQvLQQcwFZyH/CEs2/lCZBWudzjv/5MDYNnF1vwa+34p49MIJW60cDinxVfi26+tGo88OrVuxipC4MRovmE0Q3zqzlMFWWAIFIf+k+m3VrpBJvcTh/JY5WJXOTeY49k+QLygyJyFF853NwVC4ZuC00Pg0PQ+9fZeLMiwxqyhkMF2nP4x+AA5XAu+CBwcWnW3gSl4TXNBfPXXreTqvk6rySP0UcOb6qr1eW3XcK2uykg==
  • Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=colostate.edu;
<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body>
    Pols,<br>
    <br>
    Well, by actually reading the rest of the instructions in the <a 
href="https://docs.unidata.ucar.edu/tds/current/userguide/enable_tls_encryption.html";>TDS
      documentation</a> I was able to set the<span 
style="color:#000000;background-color:#ffffff;">
      certificateKeystorePassword, which fixed this problem. At least
      for website access if I push though the self-signed certificate
      warnings.<br>
      <br>
      But, pydap is failing due to the self-signed certificate and I
      haven't found a way around it yet:<br>
    </span><span style="font-family:monospace"><span 
style="color:#000000;background-color:#ffffff;">ssl.SSLCertVerificationError:
        [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
        self-signed certificate (_ssl.c:1000)</span><br>
    </span><br>
    If anyone knows a way around that please let me know.<br>
    <br>
    Thanks,<br>
    Jim<br>
    <br>
    On 7/11/24 11:47, Jim Fluke wrote:<br>
    <blockquote type="cite" 
cite="mid:aa0627d0-8aac-47ed-b691-58b2d544fe6b@xxxxxxxxxxxxx">
      
      Pols,<br>
      <br>
      I created a self-signed certificate since it's just for testing
      right now. So far I can't get it to work though. Here are the
      errors I get at TDS start up:<br>
      <span style="font-family:monospace"><span 
style="color:#000000;background-color:#ffffff;">10-Jul-2024
          15:26:16.372 SEVERE [main]
          org.apache.catalina.util.LifecycleBase.handleSubClassException
          Failed to initialize component
          [Connector[&quot;https-openssl-nio-8443&quot;]] </span><br>
        
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;org.apache.catalina.LifecycleException:
 Protocol handler
        initialization failed<br>
        &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; .<br>
        &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp; .<br>
        &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp; .<br>
        &nbsp;&nbsp;&nbsp; &nbsp;&nbsp; </span><span 
style="font-family:monospace"><span 
style="color:#000000;background-color:#ffffff;">Caused by:
          java.lang.IllegalArgumentException: Keystore was tampered
          with, or password was incorrect</span></span><span 
style="font-family:monospace"><span 
style="color:#000000;background-color:#ffffff;"><br>
          &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp; .<br>
          &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp; .<br>
          &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp; .<br>
          &nbsp;&nbsp;&nbsp; &nbsp;&nbsp; Caused by: 
java.security.UnrecoverableKeyException:
          Password verification failed</span></span><span 
style="font-family:monospace"><span 
style="color:#000000;background-color:#ffffff;"><br>
        </span></span><br>
      And, I am still using 8443. Also because this is a test
      environment.<br>
      <br>
      Do you have any idea where I can change the password. If that
      really is the problem.<br>
      <br>
      Thanks,<br>
      Jim<br>
      <br>
      <div class="moz-cite-prefix">On 7/10/24 01:33, Pols, Maarten
        wrote:<br>
      </div>
      <blockquote type="cite" 
cite="mid:AS8P195MB238656339640EC8D7C9527F1E0A42@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx">
        <meta name="Generator" content="Microsoft Word 15 (filtered medium)">
        <!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
        <style>@font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}@font-face
        {font-family:Verdana;
        panose-1:2 11 6 4 3 5 4 4 2 4;}@font-face
        {font-family:Aptos;}@font-face
        {font-family:Tahoma;
        panose-1:2 11 6 4 3 5 4 4 2 4;}@font-face
        {font-family:"Segoe UI";
        panose-1:2 11 5 2 4 2 4 2 2 3;}p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0cm;
        font-size:12.0pt;
        font-family:"Aptos",sans-serif;}a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:blue;
        text-decoration:underline;}span.E-mailStijl22
        {mso-style-type:personal-compose;
        font-family:"Tahoma",sans-serif;
        color:windowtext;
        position:relative;
        top:0pt;
        mso-text-raise:0pt;
        letter-spacing:0pt;}.MsoChpDefault
        {mso-style-type:export-only;
        font-size:10.0pt;
        mso-ligatures:none;}div.WordSection1
        {page:WordSection1;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
        <div>
          <p><span style="background-color: #feec97; color: #000;"><strong>**
                Caution: EXTERNAL Sender **</strong></span></p>
        </div>
        <div>
          <div class="WordSection1">
            <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">Dear Jim,<o:p></o:p></span></p>
            <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US"><o:p>&nbsp;</o:p></span></p>
            <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">I think you are right, first setup a SSL
                certificate, I’m also using the thredds docker image,
                together with a nginx proxy server.<o:p></o:p></span></p>
            <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US"><o:p>&nbsp;</o:p></span></p>
            <div>
              <div>
                <p class="MsoNormal"><b><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif">M.J.
                      (Maarten) Pols</span></b><b><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif"><br>
                    </span></b><b><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif">Producten
                      en services</span></b><b><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif"><br>
                    </span></b><b><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif">Systeem-
                      en applicatiebeheerder</span></b><b><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif"> 
</span></b><o:p></o:p></p>
              </div>
              <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
              <div>
                <table class="MsoNormalTable" cellpadding="0" border="0">
                  <tbody>
                    <tr>
                      <td style="padding:.75pt .75pt .75pt .75pt">
                        <p class="MsoNormal"><img 
style="width:1.8437in;height:.6354in" id="Afbeelding_x0020_2" 
src="cid:part1.x8yzqOXz.H679MeB2@colostate.edu" class="" width="177" 
height="61"><o:p></o:p></p>
                      </td>
                      <td style="padding:.75pt .75pt .75pt .75pt">
                        <div>
                          <p class="MsoNormal" 
style="margin-bottom:12.0pt"><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif">Botter
                              11-29, 8232 JN Lelystad (tevens 
postadres)</span><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif"><br>
                            </span><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif">Berkenweg
                              7, Amersfoort | Informaticalaan 8, 
Delft</span><o:p></o:p></p>
                        </div>
                        <div>
                          <p class="MsoNormal"><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif">Telefoon
                              0320 294292</span><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif"> <br>
                            </span><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif">Internet</span><span
 style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif"> <u><a 
href="http://www.hkv.nl/"; originalsrc="http://www.hkv.nl/"; 
shash="x6ho/LfPAvguAu9tDpRz/1Q7mf1eii/5xTGk2Xq8Gt48IyOuNBXPe2hbadhs4lqyAQC4TEG3YragSGRGdAntOd24HeOhjZO6AAppZ+HuqUfWU3hfeVvNr9QlvghYu9CEtmbZgzkwP7zHLLTab5hQei/B0Lb1N6zbDTDvVtMOZyE="
 moz-do-not-send="true">www.hkv.nl</a></u>
                              <o:p></o:p></span></p>
                        </div>
                      </td>
                    </tr>
                  </tbody>
                </table>
              </div>
              <div>
                <p class="MsoNormal">&nbsp;&nbsp; <o:p></o:p></p>
              </div>
            </div>
            <p class="MsoNormal"><span 
style="font-size:6.0pt;font-family:&quot;Verdana&quot;,sans-serif" 
lang="NL">HKV, de kennisondernemer voor water en
                veiligheid </span><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="NL"><o:p></o:p></span></p>
            <div>
              <div style="border:none;border-top:solid #E1E1E1 
1.0pt;padding:3.0pt 0cm 0cm 0cm">
                <p class="MsoNormal"><b><span 
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif" 
lang="NL">Van:</span></b><span 
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif" lang="NL"> 
Jim Fluke <a class="moz-txt-link-rfc2396E" 
href="mailto:james.fluke@xxxxxxxxxxxxx"; 
moz-do-not-send="true">&lt;james.fluke@xxxxxxxxxxxxx&gt;</a>
                    <br>
                    <b>Verzonden:</b> Tuesday, 9 July 2024 19:45<br>
                    <b>Aan:</b> Pols, Maarten <a class="moz-txt-link-rfc2396E" 
href="mailto:M.Pols@xxxxxx"; moz-do-not-send="true">&lt;M.Pols@xxxxxx&gt;</a>;
                    <a class="moz-txt-link-abbreviated moz-txt-link-freetext" 
href="mailto:thredds@xxxxxxxxxxxxxxxx"; 
moz-do-not-send="true">thredds@xxxxxxxxxxxxxxxx</a><br>
                    <b>Onderwerp:</b> Re: [thredds] Authentication
                    problems with the TDS and pydap<o:p></o:p></span></p>
              </div>
            </div>
            <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
            <table class="MsoNormalTable" style="width:100.0%" width="100%" 
cellspacing="0" cellpadding="0" border="0" align="left">
              <tbody>
                <tr>
                  <td style="background:#A6A6A6;padding:5.25pt 1.5pt 5.25pt 
1.5pt"><br>
                  </td>
                  <td style="width:100.0%;background:#EAEAEA;padding:5.25pt 
3.75pt 5.25pt 11.25pt" width="100%">
                    <div>
                      <p class="MsoNormal" 
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
                        <span style="font-size:9.0pt;font-family:&quot;Segoe 
UI&quot;,sans-serif;color:#212121;mso-fareast-language:NL">##
                          Let op: deze mail is afkomstig van een externe
                          afzender.</span><span style="color:black"> <a 
href="https://aka.ms/LearnAboutSenderIdentification"; 
moz-do-not-send="true"><span style="font-size:9.0pt;font-family:&quot;Segoe 
UI&quot;,sans-serif;mso-fareast-language:NL">Meer
                              informatie over waarom dit belangrijk 
is</span></a>
                        </span><o:p></o:p></p>
                    </div>
                  </td>
                  <td style="width:56.25pt;background:#EAEAEA;padding:5.25pt 
3.75pt 5.25pt 3.75pt;align:left" width="75"> <br>
                  </td>
                </tr>
              </tbody>
            </table>
            <p class="MsoNormal" style="margin-bottom:12.0pt"><br>
              <br>
              <o:p></o:p></p>
            <div>
              <p class="MsoNormal" style="margin-bottom:12.0pt">Pols,<br>
                <br>
                Thank you for your response!<br>
                <br>
                But, it still does not work. I think I probably need
                this, or something like it, but it's not enough.<br>
                <br>
                Now the web browser authentication fails with this
                message:<br>
                <span style="font-family:&quot;Courier New&quot;">Secure
                  Connection Failed<br>
                  <br>
                  An error occurred during a connection to localhost.
                  PR_END_OF_FILE_ERROR<br>
                  <br>
                  Error code: PR_END_OF_FILE_ERROR<br>
                  <br>
                  &nbsp;&nbsp;&nbsp; The page you are trying to view cannot be 
shown
                  because the authenticity of the received data could
                  not be verified.<br>
                  &nbsp;&nbsp;&nbsp; Please contact the website owners to 
inform them
                  of this problem.</span><br>
                <br>
                And the pydap authentication fails with this message:<br>
                <span style="font-family:&quot;Courier 
New&quot;;color:black;background:white">ssl.SSLEOFError:
                  [SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in
                  violation of protocol (_ssl.c:1000)</span><br>
                <br>
                Which seems to indicate that I need to add an SSL
                certificate, which I have not done. Again, I am using
                the thredds-docker image, which does not have a
                certificate by default. And the port forwarding that it
                does might be an issue as well.<br>
                <br>
                I'll try the certificate, but other suggestions would be
                very welcome.<br>
                <br>
                Jim<o:p></o:p></p>
              <div>
                <p class="MsoNormal">On 7/9/24 00:35, Pols, Maarten
                  wrote:<o:p></o:p></p>
              </div>
              <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
                <div>
                  <p><strong><span 
style="font-family:&quot;Aptos&quot;,sans-serif;color:black;background:#FEEC97">**
                        Caution: EXTERNAL Sender 
**</span></strong><o:p></o:p></p>
                </div>
                <div>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">Dear Jim,</span><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;</span><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">This problem cost me months to cover.
                      It was working in previous versions of thredds but
                      after een upgrade it broke my python 
scripts.</span><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">First of all, don’t upgrade to the
                      latest numpy packages, it will break pydap, latest
                      working version is 1.26.x</span><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;</span><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">Than to solve this issue, you need to
                      change applicationContext.xml file, this file is
                      in webapps -&gt; thredds -&gt; 
WEB-INF</span><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">You need to change line 112 and 113:</span><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;</span><o:p></o:p></p>
                  <p class="MsoNormal"><i><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;&nbsp;&nbsp; &lt;bean
                        id=&quot;restrictedDatasetAuthorizer&quot;
                        
class=&quot;thredds.servlet.restrict.TomcatAuthorizer&quot;&gt;</span></i><o:p></o:p></p>
                  <p class="MsoNormal"><i><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;property 
name=&quot;useSSL&quot;
                        value=&quot;false&quot;/&gt;</span></i><o:p></o:p></p>
                  <p class="MsoNormal"><i><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;property 
name=&quot;sslPort&quot;
                        value=&quot;8443&quot;/&gt;</span></i><o:p></o:p></p>
                  <p class="MsoNormal"><i><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;&nbsp;&nbsp; &lt;/bean&gt;</span></i><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;</span><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">Into </span><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;</span><o:p></o:p></p>
                  <p class="MsoNormal"><i><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;&nbsp;&nbsp; &lt;bean
                        id=&quot;restrictedDatasetAuthorizer&quot;
                        
class=&quot;thredds.servlet.restrict.TomcatAuthorizer&quot;&gt;</span></i><o:p></o:p></p>
                  <p class="MsoNormal"><i><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;property 
name=&quot;useSSL&quot;
                        
value=&quot;<b>true</b>&quot;/&gt;</span></i><o:p></o:p></p>
                  <p class="MsoNormal"><i><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;property 
name=&quot;sslPort&quot;
                        
value=&quot;<b>443</b>&quot;/&gt;</span></i><o:p></o:p></p>
                  <p class="MsoNormal"><i><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;&nbsp;&nbsp; &lt;/bean&gt;</span></i><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;</span><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">This was solving the issue in my
                      case, and I hope it will help you.</span><o:p></o:p></p>
                  <p class="MsoNormal"><span 
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,sans-serif;mso-fareast-language:EN-US"
 lang="EN-US">&nbsp;</span><o:p></o:p></p>
                  <div>
                    <div>
                      <div>
                        <div>
                          <p class="MsoNormal"><b><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif">M.J.
                                (Maarten) Pols<br>
                                Products and Services<br>
                                System and application administrator 
</span></b><o:p></o:p></p>
                        </div>
                        <p class="MsoNormal">&nbsp;<o:p></o:p></p>
                        <div>
                          <table class="MsoNormalTable" cellpadding="0" 
border="0">
                            <tbody>
                              <tr>
                                <td style="padding:.75pt .75pt .75pt .75pt">
                                  <p class="MsoNormal"><img 
style="width:1.8541in;height:.6354in" id="Afbeelding_x0020_1" 
src="cid:part2.YJLqE4u4.f7xPFtAc@colostate.edu" class="" width="178" 
height="61" border="0"><o:p></o:p></p>
                                </td>
                                <td style="padding:.75pt .75pt .75pt .75pt">
                                  <div>
                                    <p class="MsoNormal" 
style="margin-bottom:12.0pt"><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif">Botter
                                        11-29, 8232 JN Lelystad, The
                                        Netherlands (also postal
                                        address) <br>
                                        Berkenweg 7, Amersfoort |
                                        Informaticalaan 8, 
Delft</span><o:p></o:p></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"><span 
style="font-size:7.5pt;font-family:&quot;Verdana&quot;,sans-serif">Telephone
                                        +31 (0)320 294292 <br>
                                        Internet <u><a 
href="http://www.hkv.nl/en/"; originalsrc="http://www.hkv.nl/en/"; 
shash="EjDz5KgdCFqyzSCb1b/0qxwtzfqp6gmQZyknWMiXV4vI1rj3kZUvKRde3NvThaor3yHAZVQNJgaKfpiz8yYL+xz+y53QqpbnGowz4e3OHlrK13w4ovgb1z4D//wWZm5PevCmdq+iPQbLeaEQ7tQosbJ3gnj0UljHF1BLsXIMKxo="
 moz-do-not-send="true">www.hkv.nl/en/</a></u>
                                      </span><o:p></o:p></p>
                                  </div>
                                </td>
                              </tr>
                            </tbody>
                          </table>
                        </div>
                        <div>
                          <p class="MsoNormal">&nbsp;&nbsp; <o:p></o:p></p>
                        </div>
                        <div>
                          <p class="MsoNormal"><span 
style="font-size:6.0pt;font-family:&quot;Verdana&quot;,sans-serif" 
lang="EN-US">HKV, knowledge entrepreneurs
                              in flood risk and water resources
                              management </span><o:p></o:p></p>
                        </div>
                      </div>
                    </div>
                  </div>
                  <div>
                    <div style="border:none;border-top:solid #E1E1E1 
1.0pt;padding:3.0pt 0cm 0cm 0cm">
                      <p class="MsoNormal"><b><span 
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif" 
lang="NL">Van:</span></b><span 
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif" lang="NL"> 
thredds <a href="mailto:thredds-bounces@xxxxxxxxxxxxxxxx"; 
moz-do-not-send="true">&lt;thredds-bounces@xxxxxxxxxxxxxxxx&gt;</a>
                          <b>Namens </b>Jim Fluke<br>
                          <b>Verzonden:</b> Tuesday, 9 July 2024 00:04<br>
                          <b>Aan:</b> <a href="mailto:thredds@xxxxxxxxxxxxxxxx"; 
moz-do-not-send="true" 
class="moz-txt-link-freetext">thredds@xxxxxxxxxxxxxxxx</a><br>
                          <b>Onderwerp:</b> [thredds] Authentication
                          problems with the TDS and pydap</span><o:p></o:p></p>
                    </div>
                  </div>
                  <p class="MsoNormal">&nbsp;<o:p></o:p></p>
                  <table class="MsoNormalTable" style="width:100.0%" 
width="100%" cellspacing="0" cellpadding="0" border="0" align="left">
                    <tbody>
                      <tr>
                        <td style="background:#A6A6A6;padding:5.25pt 1.5pt 
5.25pt 1.5pt"><br>
                        </td>
                        <td 
style="width:100.0%;background:#EAEAEA;padding:5.25pt 3.75pt 5.25pt 11.25pt" 
width="100%">
                          <div>
                            <p class="MsoNormal" 
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
                              <span 
style="font-size:9.0pt;font-family:&quot;Segoe 
UI&quot;,sans-serif;color:#212121;mso-fareast-language:NL">##
                                Let op: deze mail is afkomstig van een
                                externe afzender.</span><span 
style="color:black"> <a href="https://aka.ms/LearnAboutSenderIdentification"; 
moz-do-not-send="true"><span style="font-size:9.0pt;font-family:&quot;Segoe 
UI&quot;,sans-serif;mso-fareast-language:NL">Meer
                                    informatie over waarom dit
                                    belangrijk is</span></a> 
</span><o:p></o:p></p>
                          </div>
                        </td>
                        <td 
style="width:56.25pt;background:#EAEAEA;padding:5.25pt 3.75pt 5.25pt 
3.75pt;align:left" width="75"> <br>
                        </td>
                      </tr>
                    </tbody>
                  </table>
                  <p class="MsoNormal" style="margin-bottom:12.0pt"><br>
                    <br>
                    <br>
                    <o:p></o:p></p>
                  <div>
                    <p class="MsoNormal">Hello,<br>
                      <br>
                      I'm now trying to get user authentication working
                      with our thredds-docker based TDS. I'm pretty sure
                      I have the configuration set up to enable
                      authentication as described in the TDS manual's 
&quot;<span style="color:black;background:white"><a 
href="https://docs.unidata.ucar.edu/tds/current/userguide/restict_access_to_tds.html#restrict-access-by-dataset-in-tds-catalogs";
 
originalsrc="https://docs.unidata.ucar.edu/tds/current/userguide/restict_access_to_tds.html#restrict-access-by-dataset-in-tds-catalogs";
 
shash="PZOx9FLcAuu6KJ3pOsQECLIq/7vDvB93iRpnyXJ5zfvhfz4oqoT4iw9rqvraz0aHdsiafPq6+KPO0gTalzurUvVrMsH9Ff+813xYwKGcZLlvP/amr5vxY1SYkBBN7/fssMKL9PBYZBD6DK6Ldr5iF+M3SybMn65xHalJZmVaLwU="
 moz-do-not-send="true">Restrict Access To The
                          TDS</a>&quot; page</span>. And I have verified this
                      by accessing the TDS from a browser and having the
                      credentials entry pop-up window display and work
                      correctly.<br>
                      <br>
                      But, I can't get the authentication to work in
                      Python with pydap. According to the pydap
                      documentation the credentials should be added to
                      the URL this way:<br>
                      <br>
                      <span style="font-family:&quot;Courier 
New&quot;;color:black;background:white">&gt;&gt;&gt;
                        from pydap.client import open_url </span><span 
style="font-family:&quot;Courier New&quot;"><br>
                        &gt;&gt;&gt; dataset = open_url('<a 
href="http://username:password@xxxxxxxxxxxxxxxxxx/path/to/dataset"; 
originalsrc="http://username:password@xxxxxxxxxxxxxxxxxx/path/to/dataset"; 
shash="Zh3uGJ4zUcnUVgOrhTlG7l/nXlKCe4PM7oxWe7cxzBIAQ4/9fg+bUgsHbkSIP4FoyplI22zBLF9MD+60wIcUBdKRRVkkKQrRV/Lruaq+qCaZOVMBcFWAK2BHZxAjBXo0VCy3DMM0VHC9Km/+RLtoecXyOapT33YGPxmdlARi4E4="
 
moz-do-not-send="true">http://username:password@xxxxxxxxxxxxxxxxxx/path/to/dataset</a>')<br>
                      </span><br>
                      But because <a 
href="https://docs.unidata.ucar.edu/tds/current/userguide/digested_passwords.html";
 
originalsrc="https://docs.unidata.ucar.edu/tds/current/userguide/digested_passwords.html";
 
shash="d10mi9S0MG9tBEd/3f47M085b+0at93qhvFwbqBahPl9vktr2O/lDMk0ZZsQrdNNWsQocJ915bmA4wuAKl81J2iK4IB7Mdnw9XUoeN8a2LIjbeBkgIRDyG6/gDJKpEYiIPTAGNCFXgL4gcgh1eRzRR1iMf8E4TTUEoSqA46rPb4="
 moz-do-not-send="true"> Digested Passwords</a>
                      are enabled for our TDS, it seems clear that I
                      should use the digested password, so this is what
                      I tried:<br>
                      <br>
                      <span style="font-family:&quot;Courier 
New&quot;;color:black;background:white">&gt;&gt;&gt;
                        from pydap.client import open_url </span><span 
style="font-family:&quot;Courier New&quot;"><br>
                        &gt;&gt;&gt; dataset = open_url('<span 
style="color:black;background:white">http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b</span><br>
                        <a 
href="mailto:2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf";
 moz-do-not-send="true" 
class="moz-txt-link-freetext">2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf</a><br>
                        ')<br>
                      </span><br>
                      But it does not work. Here is the output:<br>
                      <br>
                      <span style="font-family:&quot;Courier 
New&quot;;color:black;background:white">@
                        ~/devRepos/thredds-dpc-gh-actual/tests$
                        docker-compose run --rm test_opendap </span><span 
style="font-family:&quot;Courier New&quot;"><br>
                        url: <a 
href="http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b";
 moz-do-not-send="true" class="moz-txt-link-freetext">
http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b</a><br>
                        <a 
href="mailto:2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf";
 moz-do-not-send="true" 
class="moz-txt-link-freetext">2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf</a>
                        <br>
                        <br>
                        Traceback (most recent call last): <br>
                        &nbsp;File &quot;/app/opendap_pydap.py&quot;, line 8, in
                        &lt;module&gt; <br>
                        &nbsp;&nbsp;&nbsp;dataset = open_url(url) <br>
                        
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;^^^^^^^^^^^^^
 <br>
                        &nbsp;File
                        
&quot;/opt/conda/lib/python3.12/site-packages/pydap/client.py&quot;,
                        line 68, in open_url <br>
                        &nbsp;&nbsp;&nbsp;handler = 
pydap.handlers.dap.DAPHandler(url,
                        application, session, output_grid, <br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                        <br>
                        &nbsp;File
                        
&quot;/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py&quot;,
                        line 71, in __init__ <br>
                        &nbsp;&nbsp;&nbsp;self.make_dataset() <br>
                        &nbsp;File
                        
&quot;/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py&quot;,
                        line 96, in make_dataset <br>
                        &nbsp;&nbsp;&nbsp;self.dataset_from_dap2() <br>
                        &nbsp;File
                        
&quot;/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py&quot;,
                        line 109, in dataset_from_dap2 <br>
                        &nbsp;&nbsp;&nbsp;pydap.net.raise_for_status(r) <br>
                        &nbsp;File
                        
&quot;/opt/conda/lib/python3.12/site-packages/pydap/net.py&quot;,
                        line 38, in raise_for_status <br>
                        &nbsp;&nbsp;&nbsp;raise HTTPError( <br>
                        webob.exc.HTTPError: 401 Unauthorized <br>
                        &lt;!doctype html&gt;&lt;html
                        lang=&quot;en&quot;&gt;&lt;head&gt;&lt;title&gt;HTTP
                        Status 401 – Unauthorized&lt;/title&gt;&lt;style
                        type=&quot;text/css&quot;&gt;body
                        {font-family:Tahoma,Arial,sans-serif;} h1, h2,
                        h3, b {color:white;background-co<br>
                        lor:#525D76;} h1 {font-size:22px;} h2
                        {font-size:16px;} h3 {font-size:14px;} p
                        {font-size:12px;} a {color:black;} .line
{height:1px;background-color:#525D76;border:none;}&lt;/style&gt;&lt;/head&gt;&lt;bod<br>
                        y&gt;&lt;h1&gt;HTTP Status 401 –
                        Unauthorized&lt;/h1&gt;&lt;hr class=&quot;line&quot;
                        /&gt;&lt;p&gt;&lt;b&gt;Type&lt;/b&gt; Status
                        Report&lt;/p&gt;&lt;p&gt;&lt;b&gt;Description&lt;/b&gt;
                        The request has not been applied to the target
                        resource because it lacks va<br>
                        lid authentication credentials for that
                        resource.&lt;/p&gt;&lt;hr class=&quot;line&quot;
                        /&gt;&lt;h3&gt;Apache
                        Tomcat&lt;/h3&gt;&lt;/body&gt;&lt;/html&gt;<br>
                        <br>
                      </span>So, am I right to be using the digested
                      password? Do you see anything else that could be
                      wrong? Why does this work for the browser but not
                      for pydap?<br>
                      <br>
                      I will add that the algorithm for the <span 
style="color:black;background:white">
                        CredentialHandler is &quot;sha-</span><b><span 
style="color:#FF5454;background:white">512</span></b>&quot;
                      in the ~tomcat/conf/server.xml file inside the
                      container, so that is why the digested password is
                      an sha512 digest. And the clear text password is
                      &quot;flukeTmp&quot;. I'll be changing that for our
                      production system.<br>
                      <br>
                      And, all of this - the TDS configuration and the
                      test python script with the above URL - are now
                      checked in to our <a 
href="https://github.com/JimFluke/thredds-dpc/tree/master"; 
originalsrc="https://github.com/JimFluke/thredds-dpc/tree/master"; 
shash="OU/KJT9dXcykuQBuNc5fh5lBjuAHHu4Rbyqv7agsmCze+nH1SOPACKqw2usZk9XvQGHiLnfY2h+o6oFXnHRvM11Tbi21H2nCQOziEBKDUSE/JPG76iU476LFZMBCTqvJo6C22pl3+b1KGZiSX/7308eFc1/fumm5+lv8IglEpRM="
 moz-do-not-send="true">thredds-dpc</a>
                      repository on GitHub so you can look at the
                      details there.<br>
                      <br>
                      Any help would be greatly appreciated.<br>
                      <br>
                      Thanks,<br>
                      Jim<o:p></o:p></p>
                  </div>
                </div>
              </blockquote>
              <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
            </div>
          </div>
        </div>
      </blockquote>
      <br>
    </blockquote>
    <br>
  </body>
</html>

JPEG image

JPEG image

  • 2024 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: