<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
Christian,<br>
<br>
I'm now trying to use an nginx as a proxy - or maybe I should say a
reverse proxy - but I'm still having trouble. We can't publicly
expose a server here that is http, it has to be https, so I'm trying
to do things differently than what you describe. I have nginx
configured to use https and to forward requests to the TDS using
http. Here is the server block in nginx.conf:<br>
<br>
<span style="font-family:monospace"><span
style="color:#000000;background-color:#ffffff;"> server {
</span><br>
listen
443
ssl;
<br>
<br>
server_name
gcin01.cira.colostate.edu;
<br>
ssl_certificate
/root/CERTS/JUL2024/gcin01_cira_colostate_edu_cert.cer-CertOnlyPEMEncoded;<br>
ssl_certificate_key
/root/CERTS/JUL2024/gcin01.key;
<br>
<br>
location / {
<br>
proxy_pass <a
class="moz-txt-link-freetext"
href="http://localhost:7000/";>http://localhost:7000/</a>;
<br>
}
<br>
}<br>
</span><br>
The certificate settings work fine and give us an https connection
to our TDS website, but only for the catalog pages for navigating
the dataset. When I select the OpenDAP service button it gives me a
localhost:7000 DataURL for the file I'm accessing. Example:<br>
<span style="font-family:monospace"><a class="moz-txt-link-freetext"
href="http://localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2013/180/2013180111833_38146_CS_2B-GEOPROF_GRANULE_P1_R05_E06_F00.hdf";>http://localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2013/180/2013180111833_38146_CS_2B-GEOPROF_GRANULE_P1_R05_E06_F00.hdf</a><br>
<br>
</span>If I manually change this to start with <a
class="moz-txt-link-freetext" href="https://gcin01";>https://gcin01</a> then
it works fine. Is there a way to configure the OpenDAP service to
use the https start to the URL?<br>
<br>
More importantly, I can't get pydap to work through the nginx
server. When I give it a URL like this:<br>
<span style="font-family:monospace"><span
style="color:#000000;background-color:#ffffff;"><a
class="moz-txt-link-freetext"
href="https://gcin01/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2013/180/2013180111833_38146_CS_2B-GEOPROF_GRANULE_P1_R05_E06_F00.hdf";>https://gcin01/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2013/180/2013180111833_38146_CS_2B-GEOPROF_GRANULE_P1_R05_E06_F00.hdf</a></span><br>
<br>
</span>It raises and exception ending with:<br>
<span style="font-family:monospace"><span
style="color:#000000;background-color:#ffffff;">ssl.SSLCertVerificationError:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
unable to get local issuer certificate (_ssl.c:1006)<br>
<br>
</span></span>Any suggestions on how to fix this would be greatly
appreciated.<br>
<br>
Thanks,<br>
Jim<br>
<br>
<div class="moz-cite-prefix">On 7/31/24 14:49, Christian Skarby
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CA+2oyAAxv5P-hU9saWEc3cVP_gxsrKVoK7qfvLKFjmi=kpZq7w@xxxxxxxxxxxxxx">
<div>
<p><span style="background-color: #feec97; color: #000;"><strong>**
Caution: EXTERNAL Sender **</strong></span></p>
</div>
<div>
<div dir="ltr">You could run apache or nginx on port 80 (either
on the host or a separate container) - using http redirects to
https (port 443) - and have certbot running in that
container/host context (In case of container - make sure to
have the /etc/letsencrypt persisted, e.g. by using a -v
/host/path:/etc/letsencrypt - also do something to ensure that
certbot is triggered regularity to update your certs)<br>
<br>
Then run the tds container mounting -v
/host/path:/etc/letsencrypt:ro and update the configuration to
use the certificate presented by certbot.</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">ons. 31. juli 2024 kl. 01:38
skrev Jim Fluke <<a href="mailto:james.fluke@xxxxxxxxxxxxx";
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">james.fluke@xxxxxxxxxxxxx</a>>:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>Okay, we are trying to install an "official"
certificate using Certbot. Apparently, using Certbot is
now the required way to install certificates at CSU, and
we have done this successfully for Apache and ngnix
running on the host. The problem is that we are having
trouble creating a Certbot configuration that works for
the TDS running in a container (<a
href="https://github.com/Unidata/thredds-docker";
originalsrc="https://github.com/Unidata/thredds-docker";
shash="veGSyVlG2JCzxi7MNbZuLFRpwf/ck4PTvdDumU81m2aUf6Pq6y2sMRvLXch11un3gh/rhcHM0VhOQ/01FHkAaz7+/auoY1wi/ZZuvBPGvha/qV13VedQn3d1h1Mq1hKfComo802EENJuhv5+RC3w9+vpqdBF0IhXwRtW9aQ2tN0="
target="_blank" moz-do-not-send="true">thredds-docker</a>).
Has anyone done this? If so, can you share your Certbot
configuration?<br>
<br>
Thanks,<br>
Jim <br>
<br>
<div>On 7/12/24 03:45, Christian Skarby wrote:<br>
</div>
<blockquote type="cite">
<div>
<p><span
style="background-color:rgb(254,236,151);color:rgb(0,0,0)"><strong>**
Caution: EXTERNAL Sender **</strong></span></p>
</div>
<div>
<div dir="ltr"><b>The easiest and best is usually to
get a certificate from one of the renowned
suppliers generally included in the trust store of
big browsers and operating systems.<br>
</b><br>
Check out <a href="https://letsencrypt.org/";
originalsrc="https://letsencrypt.org/";
shash="bAltT8DiJdDnJq7zq4vRFqDYpbaYhVVKV529Hxwxj6HWo+ioRMKAJrKnaAg+fqlYxOR8vlKOMgg1e4++X6GgunjAb7k7HQZpbl2BYFDMZ1WIIblbdaj+Epjsv6DpmdfZodEf2YcCivADL43QmS9iaUYxIMalSkTkcu/VYodOFKk="
target="_blank" moz-do-not-send="true">https://letsencrypt.org/</a>
which provides certificates for free.<br>
If it is possible to expose the http-port (tcp/80)
of your server to the internet, that is an easy way
to start using Let's Encrypt - and if necessary they
also provides other ways to identify ownership of
hostnames, e.g. by providing DNS-records.
<div><br>
If you really would like to make your own test
certificates, you could check out <a
href="https://github.com/OpenVPN/easy-rsa";
originalsrc="https://github.com/OpenVPN/easy-rsa";
shash="h51MqAN6WPdnRqAx75M7iFDn9K9v7GfR/Mb2pplDEYgEb8E1YVdxbd9NtbWUN639rmavYkRxnP24HoZxGf8WDOrupnJE8Ca7mNhDgeVSCEej/ywhH7z6oMzhgFNHYzgfsQGo8dI1MRTvooQTcHasZuTTdY6LH1fgY/ZP07CGY6k="
target="_blank"
moz-do-not-send="true">https://github.com/OpenVPN/easy-rsa</a> <br>
The certificates are regular SSL/TLS-certificates,
and can be used for any protocol encrypted with
TLS.<br>
Read through <a
href="https://github.com/OpenVPN/easy-rsa/blob/master/doc/Intro-To-PKI.md";
originalsrc="https://github.com/OpenVPN/easy-rsa/blob/master/doc/Intro-To-PKI.md";
shash="Ra/k5ik14KreqOcMSYipYCb3ZmbmY6ljb17uUENNV9AvmcuFzMp36WnsIfByYtdG3pvbn61+yuSk+50cg5kcSFUvr2Q9m18wT1xaPHAFEXyIvBtq2/le73b41AAIuJukjntLyNZKehdPX4zlFkOfx1bxLzDTZcik6tNKy6LzrrA="
target="_blank"
moz-do-not-send="true">https://github.com/OpenVPN/easy-rsa/blob/master/doc/Intro-To-PKI.md</a>
which is an introduction to public key
infrastructure.<br>
Your users must also understand the risk of giving
you super powers: Certificate Authorities (CA) are
trusted entities in your operating system/browser,
and could issue certificates for any hostname. If
they trust your CA, you could in theory make
certificates for any existing (or non-existing
domain name) and make their browser/application
trust that site as you provide a valid certificate
issued by one of their trusted CAs. Operating a CA
also requires understanding of the trust model and
to keep track of the different certificate expiry
dates within the certificate chain from your root
certificate and down to the service certificate.<br>
<br>
Again, if possible - always use certificates from
official providers. Rolling your own CA is a big
responsibility, and not for the faint of heart</div>
<div>
<div>
<div dir="ltr" class="gmail_signature">
<div dir="ltr"><br>
--<br>
Best Regards,<br>
<br>
Christian Skarby<br>
MET Norway</div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">fre. 12. juli 2024
kl. 00:04 skrev Jim Fluke <<a
href="mailto:james.fluke@xxxxxxxxxxxxx"; target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">james.fluke@xxxxxxxxxxxxx</a>>:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>Pols,<br>
<br>
Well, by actually reading the rest of the
instructions in the <a
href="https://docs.unidata.ucar.edu/tds/current/userguide/enable_tls_encryption.html";
originalsrc="https://docs.unidata.ucar.edu/tds/current/userguide/enable_tls_encryption.html";
shash="S9aGe0HE4iJgs1o/siGTW/OOeBHJK8PdI8MSJmPENdXdXdDRoYvwYGvQ/+TN32t9JYANZHaTYA1nLyJs7n9sQysxRnRGFCCRSxb5xqcHFp2PWGXZSAcH/rZXNAsf8yCxq4vhNWP2b8/B1ai/SIvzOGgSWe28dnMdbqSr77PFhdk="
target="_blank" moz-do-not-send="true">
TDS documentation</a> I was able to set the<span
style="color:rgb(0,0,0);background-color:rgb(255,255,255)">
certificateKeystorePassword, which fixed this
problem. At least for website access if I push
though the self-signed certificate warnings.<br>
<br>
But, pydap is failing due to the self-signed
certificate and I haven't found a way around
it yet:<br>
</span><span style="font-family:monospace"><span
style="color:rgb(0,0,0);background-color:rgb(255,255,255)">ssl.SSLCertVerificationError:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate
verify failed: self-signed certificate
(_ssl.c:1000)</span><br>
</span><br>
If anyone knows a way around that please let me
know.<br>
<br>
Thanks,<br>
Jim<br>
<br>
On 7/11/24 11:47, Jim Fluke wrote:<br>
<blockquote type="cite">Pols,<br>
<br>
I created a self-signed certificate since it's
just for testing right now. So far I can't get
it to work though. Here are the errors I get
at TDS start up:<br>
<span style="font-family:monospace"><span
style="color:rgb(0,0,0);background-color:rgb(255,255,255)">10-Jul-2024
15:26:16.372 SEVERE [main]
org.apache.catalina.util.LifecycleBase.handleSubClassException
Failed to initialize component
[Connector["https-openssl-nio-8443"]]
</span><br>
org.apache.catalina.LifecycleException:
Protocol handler
initialization failed<br>
.<br>
.<br>
.<br>
</span><span
style="font-family:monospace"><span
style="color:rgb(0,0,0);background-color:rgb(255,255,255)">Caused by:
java.lang.IllegalArgumentException:
Keystore was tampered with, or password
was incorrect</span></span><span
style="font-family:monospace"><span
style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br>
.<br>
.<br>
.<br>
Caused by:
java.security.UnrecoverableKeyException:
Password verification failed</span></span><span
style="font-family:monospace"><span
style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br>
</span></span><br>
And, I am still using 8443. Also because this
is a test environment.<br>
<br>
Do you have any idea where I can change the
password. If that really is the problem.<br>
<br>
Thanks,<br>
Jim<br>
<br>
<div>On 7/10/24 01:33, Pols, Maarten wrote:<br>
</div>
<blockquote type="cite">
<div>
<p><span
style="background-color:rgb(254,236,151);color:rgb(0,0,0)"><strong>**
Caution: EXTERNAL Sender
**</strong></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">Dear
Jim,</span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">I
think you are right, first setup a
SSL certificate, I’m also using the
thredds docker image, together with
a nginx proxy server.</span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US"> </span></p>
<div>
<div>
<p class="MsoNormal"><b><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">M.J. (Maarten)
Pols</span></b><b><span
style="font-size:7.5pt;font-family:Verdana,sans-serif"><br>
</span></b><b><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">Producten en
services</span></b><b><span
style="font-size:7.5pt;font-family:Verdana,sans-serif"><br>
</span></b><b><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">Systeem- en
applicatiebeheerder</span></b><b><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">
</span></b></p>
</div>
<p class="MsoNormal"> </p>
<div>
<table cellpadding="0" border="0">
<tbody>
<tr>
<td style="padding:0.75pt">
<p class="MsoNormal"><img
style="width: 1.8437in; height: 0.6354in;"
id="m_1205346719512352428m_1981048943580741646m_9035996778467414347Afbeelding_x0020_2"
src="cid:part1.l20ATOvf.lsJu1XXD@colostate.edu" width="177" height="61"
class=""></p>
</td>
<td style="padding:0.75pt">
<div>
<p class="MsoNormal"
style="margin-bottom:12pt"><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">Botter 11-29,
8232 JN Lelystad
(tevens
postadres)</span><span
style="font-size:7.5pt;font-family:Verdana,sans-serif"><br>
</span><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">Berkenweg 7,
Amersfoort |
Informaticalaan 8,
Delft</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">Telefoon 0320
294292</span><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">
<br>
</span><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">Internet</span><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">
<u><a
href="http://www.hkv.nl/"; originalsrc="http://www.hkv.nl/";
shash="NzQjXFflRepy+D1XThNobOWz319GuW84LUq1b4lEORN7dKWN1EeJ9RHX7cEUXPZytgVnPPy+z0Uuw78/saGQ/b17ehCFCh8l9bc4LXMTXC/Bj+GP3MVkhsQsoyQyDGwWSbuKzgMq3Hev83aJLPjPehERWoVGitKdF1bcb8pH1jA="
target="_blank" moz-do-not-send="true">www.hkv.nl</a></u> </span></p>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
</div>
<p class="MsoNormal"><span
style="font-size:6pt;font-family:Verdana,sans-serif" lang="NL">HKV, de
kennisondernemer voor water en
veiligheid
</span><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="NL"></span></p>
<div>
<div
style="border-right:none;border-bottom:none;border-left:none;border-top:1pt
solid rgb(225,225,225);padding:3pt 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:11pt;font-family:Calibri,sans-serif"
lang="NL">Van:</span></b><span
style="font-size:11pt;font-family:Calibri,sans-serif" lang="NL"> Jim
Fluke
<a
href="mailto:james.fluke@xxxxxxxxxxxxx"; target="_blank"
moz-do-not-send="true"><james.fluke@xxxxxxxxxxxxx></a>
<br>
<b>Verzonden:</b> Tuesday, 9
July 2024 19:45<br>
<b>Aan:</b> Pols, Maarten <a
href="mailto:M.Pols@xxxxxx"; target="_blank"
moz-do-not-send="true"><M.Pols@xxxxxx></a>;
<a
href="mailto:thredds@xxxxxxxxxxxxxxxx"; target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">thredds@xxxxxxxxxxxxxxxx</a><br>
<b>Onderwerp:</b> Re: [thredds]
Authentication problems with the
TDS and pydap</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<table style="width:100%" width="100%"
cellspacing="0" cellpadding="0" border="0" align="left">
<tbody>
<tr>
<td
style="background:rgb(166,166,166);padding:5.25pt 1.5pt"><br>
</td>
<td
style="width:100%;background:rgb(234,234,234);padding:5.25pt 3.75pt 5.25pt
11.25pt" width="100%">
<div>
<p class="MsoNormal"><span
style="font-size:9pt;font-family:"Segoe
UI",sans-serif;color:rgb(33,33,33)">##
Let op: deze mail is
afkomstig van een externe
afzender.</span><span
style="color:black">
<a
href="https://aka.ms/LearnAboutSenderIdentification"; target="_blank"
moz-do-not-send="true"><span style="font-size:9pt;font-family:"Segoe
UI",sans-serif">Meer
informatie over waarom
dit belangrijk is</span></a>
</span></p>
</div>
</td>
<td
style="width:56.25pt;background:rgb(234,234,234);padding:5.25pt 3.75pt"
width="75">
<br>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"
style="margin-bottom:12pt"><br>
<br>
</p>
<div>
<p class="MsoNormal"
style="margin-bottom:12pt">Pols,<br>
<br>
Thank you for your response!<br>
<br>
But, it still does not work. I think
I probably need this, or something
like it, but it's not enough.<br>
<br>
Now the web browser authentication
fails with this message:<br>
<span style="font-family:"Courier
New"">Secure Connection Failed<br>
<br>
An error occurred during a
connection to localhost.
PR_END_OF_FILE_ERROR<br>
<br>
Error code: PR_END_OF_FILE_ERROR<br>
<br>
The page you are
trying to
view cannot be shown because the
authenticity of the received data
could not be verified.<br>
Please contact the
website
owners to inform them of this
problem.</span><br>
<br>
And the pydap authentication fails
with this message:<br>
<span style="font-family:"Courier
New";color:black;background:white">ssl.SSLEOFError:
[SSL:
UNEXPECTED_EOF_WHILE_READING] EOF
occurred in violation of protocol
(_ssl.c:1000)</span><br>
<br>
Which seems to indicate that I need
to add an SSL certificate, which I
have not done. Again, I am using the
thredds-docker image, which does not
have a certificate by default. And
the port forwarding that it does
might be an issue as well.<br>
<br>
I'll try the certificate, but other
suggestions would be very welcome.<br>
<br>
Jim</p>
<div>
<p class="MsoNormal">On 7/9/24
00:35, Pols, Maarten wrote:</p>
</div>
<blockquote
style="margin-top:5pt;margin-bottom:5pt">
<div>
<p><strong><span
style="font-family:Aptos,sans-serif;color:black;background:rgb(254,236,151)">**
Caution: EXTERNAL Sender
**</span></strong></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">Dear
Jim,</span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">This
problem cost me months to
cover. It was working in
previous versions of thredds
but after een upgrade it broke
my python scripts.</span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">First
of all, don’t upgrade to the
latest numpy packages, it will
break pydap, latest working
version is 1.26.x</span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">Than
to solve this issue, you need
to change
applicationContext.xml file,
this file is in webapps ->
thredds -> WEB-INF</span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">You
need to change line 112 and
113:</span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US"> </span></p>
<p class="MsoNormal"><i><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US">
<bean
id="restrictedDatasetAuthorizer"
class="thredds.servlet.restrict.TomcatAuthorizer"></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US">
<property name="useSSL"
value="false"/></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US">
<property
name="sslPort"
value="8443"/></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US">
</bean></span></i></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">Into
</span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US"> </span></p>
<p class="MsoNormal"><i><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US">
<bean
id="restrictedDatasetAuthorizer"
class="thredds.servlet.restrict.TomcatAuthorizer"></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US">
<property name="useSSL"
value="<b>true</b>"/></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US">
<property
name="sslPort"
value="<b>443</b>"/></span></i></p>
<p class="MsoNormal"><i><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US">
</bean></span></i></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">This
was solving the issue in my
case, and I hope it will help
you.</span></p>
<p class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US"> </span></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">M.J. (Maarten)
Pols<br>
Products and
Services<br>
System and
application
administrator
</span></b></p>
</div>
<p class="MsoNormal"> </p>
<div>
<table cellpadding="0" border="0">
<tbody>
<tr>
<td style="padding:0.75pt">
<p class="MsoNormal"><img
style="width: 1.8541in; height: 0.6354in;"
id="m_1205346719512352428m_1981048943580741646m_9035996778467414347Afbeelding_x0020_1"
src="cid:part2.yId1YdGV.ORvSaL67@colostate.edu" width="178" height="61"
border="0" class=""></p>
</td>
<td style="padding:0.75pt">
<div>
<p class="MsoNormal"
style="margin-bottom:12pt"><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">Botter 11-29,
8232 JN
Lelystad, The
Netherlands
(also postal
address)
<br>
Berkenweg 7,
Amersfoort |
Informaticalaan
8, Delft</span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:7.5pt;font-family:Verdana,sans-serif">Telephone +31
(0)320 294292
<br>
Internet <u><a
href="http://www.hkv.nl/en/"; originalsrc="http://www.hkv.nl/en/";
shash="GvKgyi7dPfSA+8gBTewGTihOzoqo9bCUvk6x5PVjwHyevruKQQGa+PUoiPCgjhPWbVqucHqhgP3MEka4viU1wSb5+nsArv6XqkpHln40bnsc2Ifv2DUhDld0EvPGgJGXXFX98YwgGVQTlovcfXEJcW2/IqJejPKkvnh9u25hGpY="
target="_blank" moz-do-not-send="true">www.hkv.nl/en/</a></u>
</span></p>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<div>
<p class="MsoNormal">
</p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:6pt;font-family:Verdana,sans-serif" lang="EN-US">HKV,
knowledge
entrepreneurs in flood
risk and water
resources management
</span></p>
</div>
</div>
</div>
</div>
<div>
<div
style="border-right:none;border-bottom:none;border-left:none;border-top:1pt
solid rgb(225,225,225);padding:3pt 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:11pt;font-family:Calibri,sans-serif"
lang="NL">Van:</span></b><span
style="font-size:11pt;font-family:Calibri,sans-serif" lang="NL"> thredds
<a
href="mailto:thredds-bounces@xxxxxxxxxxxxxxxx"; target="_blank"
moz-do-not-send="true"><thredds-bounces@xxxxxxxxxxxxxxxx></a>
<b>Namens </b>Jim Fluke<br>
<b>Verzonden:</b> Tuesday,
9 July 2024 00:04<br>
<b>Aan:</b> <a
href="mailto:thredds@xxxxxxxxxxxxxxxx"; target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">thredds@xxxxxxxxxxxxxxxx</a><br>
<b>Onderwerp:</b>
[thredds] Authentication
problems with the TDS and
pydap</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<table style="width:100%" width="100%"
cellspacing="0" cellpadding="0" border="0" align="left">
<tbody>
<tr>
<td
style="background:rgb(166,166,166);padding:5.25pt 1.5pt"><br>
</td>
<td
style="width:100%;background:rgb(234,234,234);padding:5.25pt 3.75pt 5.25pt
11.25pt" width="100%">
<div>
<p class="MsoNormal"><span
style="font-size:9pt;font-family:"Segoe
UI",sans-serif;color:rgb(33,33,33)">##
Let op: deze mail is
afkomstig van een
externe
afzender.</span><span style="color:black">
<a
href="https://aka.ms/LearnAboutSenderIdentification"; target="_blank"
moz-do-not-send="true"><span style="font-size:9pt;font-family:"Segoe
UI",sans-serif">Meer
informatie over
waarom dit
belangrijk is</span></a>
</span></p>
</div>
</td>
<td
style="width:56.25pt;background:rgb(234,234,234);padding:5.25pt 3.75pt"
width="75">
<br>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"
style="margin-bottom:12pt"><br>
<br>
<br>
</p>
<div>
<p class="MsoNormal">Hello,<br>
<br>
I'm now trying to get user
authentication working with
our thredds-docker based TDS.
I'm pretty sure I have the
configuration set up to enable
authentication as described in
the TDS manual's "<span
style="color:black;background:white"><a
href="https://docs.unidata.ucar.edu/tds/current/userguide/restict_access_to_tds.html#restrict-access-by-dataset-in-tds-catalogs";
originalsrc="https://docs.unidata.ucar.edu/tds/current/userguide/restict_access_to_tds.html#restrict-access-by-dataset-in-tds-catalogs";
shash="Drb9fSYqjRfCVbn1XbE57gmAWLItjGYQ396p4wXbfvAJuYq8Fnvp/cXkjFQAbbOZV8hGNPs7Jkbavd8Y+iJLtFbfaMWcK7dKDLL7Y3T2swVpZMVePq14mqsHAjUmb3uoHzYpuBMJlT8nCRIZLiQ8f0+NyWAz/dtVbsvHhSDAYZI="
target="_blank" moz-do-not-send="true">Restrict
Access To The TDS</a>"
page</span>. And I have
verified this by accessing the
TDS from a browser and having
the credentials entry pop-up
window display and work
correctly.<br>
<br>
But, I can't get the
authentication to work in
Python with pydap. According
to the pydap documentation the
credentials should be added to
the URL this way:<br>
<br>
<span
style="font-family:"Courier
New";color:black;background:white">>>>
from pydap.client import
open_url
</span><span
style="font-family:"Courier New""><br>
>>> dataset =
open_url('<a
href="http://username:password@xxxxxxxxxxxxxxxxxx/path/to/dataset";
originalsrc="http://username:password@xxxxxxxxxxxxxxxxxx/path/to/dataset";
shash="C6tRWje7IumZnX94v7we7V8ATaWsCfs5J9n/kcp+IY9kJMsvKB/ERzvejgLKFtbkFfMSG/AwJYT/5wA95oJ8Ogr+1Q0w0wUmLoerUrzxtIMJ48XfEhtp5UvhAOnOAOGPEa4nxwj1I39MMjeLPjk24o4Wv6HUosLoSuUzSsmWOsM="
target="_blank"
moz-do-not-send="true">http://username:password@xxxxxxxxxxxxxxxxxx/path/to/dataset</a>')<br>
</span><br>
But because <a
href="https://docs.unidata.ucar.edu/tds/current/userguide/digested_passwords.html";
originalsrc="https://docs.unidata.ucar.edu/tds/current/userguide/digested_passwords.html";
shash="u4nOO4wPM1jJrcTvoSKEtUCu7+ABrCSYARoSbyAHcVUj+ab/z2s+7Bl/havHasavmN4rslT3NX7gBvgKFgLjFSHRn4LkXLsNgF8HwqkGD4wHK5jwtPfUGu2RA4ZN3yRd6h+7LUaayY2f87u6l0o1IzapI6XJcyBg5q6ADhTRSZw="
target="_blank" moz-do-not-send="true">
Digested Passwords</a> are
enabled for our TDS, it seems
clear that I should use the
digested password, so this is
what I tried:<br>
<br>
<span
style="font-family:"Courier
New";color:black;background:white">>>>
from pydap.client import
open_url
</span><span
style="font-family:"Courier New""><br>
>>> dataset =
open_url('<span
style="color:black;background:white">http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b</span><br>
<a
href="mailto:2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf";
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf</a><br>
')<br>
</span><br>
But it does not work. Here is
the output:<br>
<br>
<span
style="font-family:"Courier New";color:black;background:white">@
~/devRepos/thredds-dpc-gh-actual/tests$ docker-compose run --rm
test_opendap
</span><span
style="font-family:"Courier New""><br>
url: <a
href="http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b";
target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">
http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b</a><br>
<a
href="mailto:2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf";
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf</a>
<br>
<br>
Traceback (most recent call
last): <br>
File
"/app/opendap_pydap.py",
line 8, in <module> <br>
dataset =
open_url(url) <br>
^^^^^^^^^^^^^
<br>
File
"/opt/conda/lib/python3.12/site-packages/pydap/client.py",
line 68, in open_url
<br>
handler =
pydap.handlers.dap.DAPHandler(url,
application, session,
output_grid,
<br>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<br>
File
"/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py",
line 71, in __init__
<br>
self.make_dataset() <br>
File
"/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py",
line 96, in make_dataset
<br>
self.dataset_from_dap2()
<br>
File
"/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py",
line 109, in
dataset_from_dap2
<br>
pydap.net.raise_for_status(r) <br>
File
"/opt/conda/lib/python3.12/site-packages/pydap/net.py",
line 38, in raise_for_status
<br>
raise HTTPError(
<br>
webob.exc.HTTPError: 401
Unauthorized <br>
<!doctype
html><html
lang="en"><head><title>HTTP
Status 401 –
Unauthorized</title><style
type="text/css">body
{font-family:Tahoma,Arial,sans-serif;}
h1, h2, h3, b
{color:white;background-co<br>
lor:#525D76;} h1
{font-size:22px;} h2
{font-size:16px;} h3
{font-size:14px;} p
{font-size:12px;} a
{color:black;} .line
{height:1px;background-color:#525D76;border:none;}</style></head><bod<br>
y><h1>HTTP Status
401 –
Unauthorized</h1><hr
class="line"
/><p><b>Type</b>
Status
Report</p><p><b>Description</b>
The request has not been
applied to the target
resource because it lacks va<br>
lid authentication
credentials for that
resource.</p><hr
class="line"
/><h3>Apache
Tomcat</h3></body></html><br>
<br>
</span>So, am I right to be
using the digested password?
Do you see anything else that
could be wrong? Why does this
work for the browser but not
for pydap?<br>
<br>
I will add that the algorithm
for the <span
style="color:black;background:white">
CredentialHandler is
"sha-</span><b><span
style="color:rgb(255,84,84);background:white">512</span></b>" in the
~tomcat/conf/server.xml file
inside the container, so that
is why the digested password
is an sha512 digest. And the
clear text password is
"flukeTmp". I'll be changing
that for our production
system.<br>
<br>
And, all of this - the TDS
configuration and the test
python script with the above
URL - are now checked in to
our
<a
href="https://github.com/JimFluke/thredds-dpc/tree/master";
originalsrc="https://github.com/JimFluke/thredds-dpc/tree/master";
shash="xt/ht8KVus2zk4uamb//Z5qCnDMkGk87hbrOiexpWxCguiQvqOSsoDJBZrV0Jinw5sKiUnP8BuZFZCS8DVbZoTxXpmYmC6U3p1jbn5zLNK6xd4nsh0sixTZSDdXjo5Wkgbv075m38MUnVi/fqwdA4DtcVexwRHzMCGYR94XIrbc="
target="_blank" moz-do-not-send="true">thredds-dpc</a>
repository on GitHub so you
can look at the details there.<br>
<br>
Any help would be greatly
appreciated.<br>
<br>
Thanks,<br>
Jim</p>
</div>
</div>
</blockquote>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</blockquote>
<br>
</blockquote>
<br>
</div>
_______________________________________________<br>
NOTE: All exchanges posted to Unidata maintained
email lists are<br>
recorded in the Unidata inquiry tracking system
and made publicly<br>
available through the web. Users who post to any
of the lists we<br>
maintain are reminded to remove any personal
information that they<br>
do not want to be made public.<br>
<br>
<br>
thredds mailing list<br>
<a href="mailto:thredds@xxxxxxxxxxxxxxxx"; target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">thredds@xxxxxxxxxxxxxxxx</a><br>
For list information or to unsubscribe, visit: <a
href="https://www.unidata.ucar.edu/mailing_lists/";
originalsrc="https://www.unidata.ucar.edu/mailing_lists/";
shash="nkX8HQdlKMY4Swz7Gmwc0/JwJ5kn6EBB5H20H6bPLZoJB9RJ4BhTV2TKwG/bB5QUyall4UouNt1ss9J5OAI3G6MbqnpLJgltw7APgTJfOHhpSBtrAUqhV7V66BLLdDKZGyGPrdu2CuXCRIwURaEFDdbMg726GtrJA6QKxLdxweU="
rel="noreferrer" target="_blank" moz-do-not-send="true">
https://www.unidata.ucar.edu/mailing_lists/</a>
<br>
</blockquote>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</body>
</html>
