<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
Pols,<br>
<br>
Thank you for your response!<br>
<br>
But, it still does not work. I think I probably need this, or
something like it, but it's not enough.<br>
<br>
Now the web browser authentication fails with this message:<br>
<font face="monospace">Secure Connection Failed<br>
<br>
An error occurred during a connection to localhost.
PR_END_OF_FILE_ERROR<br>
<br>
Error code: PR_END_OF_FILE_ERROR<br>
<br>
The page you are trying to view cannot be shown
because the
authenticity of the received data could not be verified.<br>
Please contact the website owners to inform them of
this
problem.</font><br>
<br>
And the pydap authentication fails with this message:<br>
<span style="font-family:monospace"><span
style="color:#000000;background-color:#ffffff;">ssl.SSLEOFError:
[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of
protocol (_ssl.c:1000)</span></span><br>
<br>
Which seems to indicate that I need to add an SSL certificate, which
I have not done. Again, I am using the thredds-docker image, which
does not have a certificate by default. And the port forwarding that
it does might be an issue as well.<br>
<br>
I'll try the certificate, but other suggestions would be very
welcome.<br>
<br>
Jim<br>
<br>
<div class="moz-cite-prefix">On 7/9/24 00:35, Pols, Maarten wrote:<br>
</div>
<blockquote type="cite"
cite="mid:AS8P195MB2386AFC9DDB7809B7E43419BE0DB2@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}@font-face
{font-family:Aptos;}@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}span.E-mailStijl20
{mso-style-type:personal-compose;
font-family:"Tahoma",sans-serif;
color:windowtext;
position:relative;
top:0pt;
mso-text-raise:0pt;
letter-spacing:0pt;}.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}div.WordSection1
{page:WordSection1;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div>
<p><span style="background-color: #feec97; color: #000;"><strong>**
Caution: EXTERNAL Sender **</strong></span></p>
</div>
<div>
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US">Dear Jim,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US">This problem cost me months to cover. It was
working in previous versions of thredds but after een
upgrade it broke my python scripts.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US">First of all, don’t upgrade to the latest
numpy packages, it will break pydap, latest working
version is 1.26.x<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US">Than to solve this issue, you need to change
applicationContext.xml file, this file is in webapps ->
thredds -> WEB-INF<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US">You need to change line 112 and 113:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><i><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"> <bean
id="restrictedDatasetAuthorizer"
class="thredds.servlet.restrict.TomcatAuthorizer"><o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"> <property
name="useSSL"
value="false"/><o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"> <property
name="sslPort"
value="8443"/><o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"> </bean><o:p></o:p></span></i></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US">Into
<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><i><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"> <bean
id="restrictedDatasetAuthorizer"
class="thredds.servlet.restrict.TomcatAuthorizer"><o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"> <property
name="useSSL"
value="<b>true</b>"/><o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"> <property
name="sslPort"
value="<b>443</b>"/><o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"> </bean><o:p></o:p></span></i></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US">This was solving the issue in my case, and I
hope it will help you.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"><o:p> </o:p></span></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif">M.J.
(Maarten) Pols</span></b><b><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif"><br>
</span></b><b><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif">Products
and Services</span></b><b><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif"><br>
</span></b><b><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif">System
and application administrator</span></b><b><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif">
</span></b><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<table class="MsoNormalTable" cellpadding="0" border="0">
<tbody>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt">
<p class="MsoNormal"><img
style="width:1.8541in;height:.6354in" id="Afbeelding_x0020_1"
src="cid:part1.1rvSaRHY.OSR8l9YS@colostate.edu" class="" width="178"
height="61"><o:p></o:p></p>
</td>
<td style="padding:.75pt .75pt .75pt .75pt">
<div>
<p class="MsoNormal"
style="margin-bottom:12.0pt"><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif">Botter
11-29, 8232 JN Lelystad, The Netherlands
(also postal address)</span><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif">
<br>
</span><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif">Berkenweg
7, Amersfoort | Informaticalaan 8,
Delft</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif">Telephone
+31 (0)320 294292</span><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif">
<br>
</span><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif">Internet</span><span
style="font-size:7.5pt;font-family:"Verdana",sans-serif">
<u><a href="http://www.hkv.nl/en/";
originalsrc="http://www.hkv.nl/en/";
shash="As/YnprnkcnshR0Zp5LvK1J7OcN0f+o2ZLDwEyVvZShvY67IDH16q5QIC+sr/FNdHmqWJUrDSNPlDtuwZtfzl+NQcXhhEFacmsGIgwab06eRHDdj1kzHUoPSJkK+RXcHWXOouRSk4iO+THWB3p9NXgs0ikq88h4ky8tbJ0ctPyU="
moz-do-not-send="true">www.hkv.nl/en/</a></u>
<o:p></o:p></span></p>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:6.0pt;font-family:"Verdana",sans-serif"
lang="EN-US">HKV, knowledge entrepreneurs in flood
risk and water resources management
</span><span
style="font-size:10.0pt;font-family:"Tahoma",sans-serif;mso-fareast-language:EN-US"
lang="EN-US"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif"
lang="NL">Van:</span></b><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif" lang="NL">
thredds
<a class="moz-txt-link-rfc2396E"
href="mailto:thredds-bounces@xxxxxxxxxxxxxxxx";><thredds-bounces@xxxxxxxxxxxxxxxx></a>
<b>Namens </b>Jim Fluke<br>
<b>Verzonden:</b> Tuesday, 9 July 2024 00:04<br>
<b>Aan:</b> <a class="moz-txt-link-abbreviated"
href="mailto:thredds@xxxxxxxxxxxxxxxx";>thredds@xxxxxxxxxxxxxxxx</a><br>
<b>Onderwerp:</b> [thredds] Authentication problems
with the TDS and pydap<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<table class="MsoNormalTable" style="width:100.0%" width="100%"
cellspacing="0" cellpadding="0" border="0" align="left">
<tbody>
<tr>
<td style="background:#A6A6A6;padding:5.25pt 1.5pt 5.25pt
1.5pt"><br>
</td>
<td style="width:100.0%;background:#EAEAEA;padding:5.25pt
3.75pt 5.25pt 11.25pt" width="100%">
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<span style="font-size:9.0pt;font-family:"Segoe
UI",sans-serif;color:#212121;mso-fareast-language:NL">##
Let op: deze mail is afkomstig van een externe
afzender.</span><span style="color:black">
<a href="https://aka.ms/LearnAboutSenderIdentification";
moz-do-not-send="true"><span style="font-size:9.0pt;font-family:"Segoe
UI",sans-serif;mso-fareast-language:NL">Meer
informatie over waarom dit belangrijk is</span></a>
</span><o:p></o:p></p>
</div>
</td>
<td style="width:56.25pt;background:#EAEAEA;padding:5.25pt
3.75pt 5.25pt 3.75pt;align:left" width="75">
<br>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
<br>
<o:p></o:p></p>
<div>
<p class="MsoNormal">Hello,<br>
<br>
I'm now trying to get user authentication working with our
thredds-docker based TDS. I'm pretty sure I have the
configuration set up to enable authentication as described
in the TDS manual's "<span
style="color:black;background:white"><a
href="https://docs.unidata.ucar.edu/tds/current/userguide/restict_access_to_tds.html#restrict-access-by-dataset-in-tds-catalogs";
originalsrc="https://docs.unidata.ucar.edu/tds/current/userguide/restict_access_to_tds.html#restrict-access-by-dataset-in-tds-catalogs";
shash="pZcl0sPaVUPmHHqEb/rneSoRRL1nZW4ehKj6xe+8r9XmaVuVkP0Psu409fzHoMpJZNFLn7xUr4zvkiDBEIcFGbq+UtsF5ATh9Rb/C6Mcs2YcdO0N4pSpHAHYvRX6D2isfAV6jye2zKBTuwXWu4GFAGP/czrhZ2fbbN0EH0aCUeY="
moz-do-not-send="true">Restrict Access To The TDS</a>"
page</span>. And I have verified this by accessing the
TDS from a browser and having the credentials entry pop-up
window display and work correctly.<br>
<br>
But, I can't get the authentication to work in Python with
pydap. According to the pydap documentation the
credentials should be added to the URL this way:<br>
<br>
<span style="font-family:"Courier
New";color:black;background:white">>>>
from pydap.client import open_url
</span><span style="font-family:"Courier New""><br>
>>> dataset = open_url('<a
href="http://username:password@xxxxxxxxxxxxxxxxxx/path/to/dataset";
originalsrc="http://username:password@xxxxxxxxxxxxxxxxxx/path/to/dataset";
shash="cTzPKUQifNYs75zqNHPhLAMdahL2YOb8EJpedcjMkR456e2JgIDDlVJLSpZ6yjQRQYeRkV5aA5A5EWHNIRFUCuv8SI/547YSr7tbEy9YAaHAgwE4I9yzIdv5GU1WgszA7of1y5Y3B6r9Lf7WuY2SPa5wDPIRARHEjW0Emy11wVk="
moz-do-not-send="true">http://username:password@xxxxxxxxxxxxxxxxxx/path/to/dataset</a>')<br>
</span><br>
But because <a
href="https://docs.unidata.ucar.edu/tds/current/userguide/digested_passwords.html";
originalsrc="https://docs.unidata.ucar.edu/tds/current/userguide/digested_passwords.html";
shash="u5Vt+xvgNVY7Qdw4jdb46FDhE2VmHdCO6UBys+h6+dtDvYuwuohR1pBqcoR0WntObzHZtFjWBRQTCMcnPprrbq31xH7Sh3nFFN7i8O6g8wQEocexstF8QV9/GwOE8cCDAIu7xZZmI2ie8JalcYCoSCja4Vyxx/BZD60mUNwNg34="
moz-do-not-send="true">
Digested Passwords</a> are enabled for our TDS, it seems
clear that I should use the digested password, so this is
what I tried:<br>
<br>
<span style="font-family:"Courier
New";color:black;background:white">>>>
from pydap.client import open_url
</span><span style="font-family:"Courier New""><br>
>>> dataset = open_url('<span
style="color:black;background:white">http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b</span><br>
<a
href="mailto:2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf";
moz-do-not-send="true"
class="moz-txt-link-freetext">2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf</a><br>
')<br>
</span><br>
But it does not work. Here is the output:<br>
<br>
<span style="font-family:"Courier
New";color:black;background:white">@
~/devRepos/thredds-dpc-gh-actual/tests$ docker-compose
run --rm test_opendap
</span><span style="font-family:"Courier New""><br>
url: <a
href="http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b";
moz-do-not-send="true" class="moz-txt-link-freetext">
http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b</a><br>
<a
href="mailto:2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf";
moz-do-not-send="true"
class="moz-txt-link-freetext">2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf</a>
<br>
<br>
Traceback (most recent call last): <br>
File "/app/opendap_pydap.py", line 8, in
<module>
<br>
dataset = open_url(url) <br>
^^^^^^^^^^^^^
<br>
File
"/opt/conda/lib/python3.12/site-packages/pydap/client.py",
line 68, in open_url
<br>
handler = pydap.handlers.dap.DAPHandler(url,
application, session, output_grid,
<br>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<br>
File
"/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py",
line 71, in __init__
<br>
self.make_dataset() <br>
File
"/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py",
line 96, in make_dataset
<br>
self.dataset_from_dap2() <br>
File
"/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py",
line 109, in dataset_from_dap2
<br>
pydap.net.raise_for_status(r) <br>
File
"/opt/conda/lib/python3.12/site-packages/pydap/net.py",
line 38, in raise_for_status
<br>
raise HTTPError( <br>
webob.exc.HTTPError: 401 Unauthorized <br>
<!doctype html><html
lang="en"><head><title>HTTP Status 401
–
Unauthorized</title><style
type="text/css">body
{font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b
{color:white;background-co<br>
lor:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;}
h3 {font-size:14px;} p {font-size:12px;} a
{color:black;} .line
{height:1px;background-color:#525D76;border:none;}</style></head><bod<br>
y><h1>HTTP Status 401 –
Unauthorized</h1><hr class="line"
/><p><b>Type</b> Status
Report</p><p><b>Description</b>
The request has not been applied to the target resource
because it lacks va<br>
lid authentication credentials for that
resource.</p><hr class="line"
/><h3>Apache
Tomcat</h3></body></html><br>
<br>
</span>So, am I right to be using the digested password?
Do you see anything else that could be wrong? Why does
this work for the browser but not for pydap?<br>
<br>
I will add that the algorithm for the <span
style="color:black;background:white">
CredentialHandler is "sha-</span><b><span
style="color:#FF5454;background:white">512</span></b>"
in the ~tomcat/conf/server.xml file inside the container,
so that is why the digested password is an sha512 digest.
And the clear text password is "flukeTmp". I'll be
changing that for our production system.<br>
<br>
And, all of this - the TDS configuration and the test
python script with the above URL - are now checked in to
our
<a href="https://github.com/JimFluke/thredds-dpc/tree/master";
originalsrc="https://github.com/JimFluke/thredds-dpc/tree/master";
shash="lhskc/a/dSPDwMU9Ya5a1uuudXtWJ1fn1uUXnlD2nSguI5bT8lG9RSRikJK8pSt509/3Wi++7m3GVpvCGfZirR4fcsVDddumF8JJCFMHwl3j6p9DvaKFgva5xxp12OF9VGi+oxWwBW4JzSvvJk3YWmw9gcBUs1uMV9pJcM+PSZg="
moz-do-not-send="true">thredds-dpc</a> repository on
GitHub so you can look at the details there.<br>
<br>
Any help would be greatly appreciated.<br>
<br>
Thanks,<br>
Jim<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>
