Dear Jim,
This problem cost me months to cover. It was working in previous versions of
thredds but after een upgrade it broke my python scripts.
First of all, don’t upgrade to the latest numpy packages, it will break pydap,
latest working version is 1.26.x
Than to solve this issue, you need to change applicationContext.xml file, this
file is in webapps -> thredds -> WEB-INF
You need to change line 112 and 113:
<bean id="restrictedDatasetAuthorizer"
class="thredds.servlet.restrict.TomcatAuthorizer">
<property name="useSSL" value="false"/>
<property name="sslPort" value="8443"/>
</bean>
Into
<bean id="restrictedDatasetAuthorizer"
class="thredds.servlet.restrict.TomcatAuthorizer">
<property name="useSSL" value="true"/>
<property name="sslPort" value="443"/>
</bean>
This was solving the issue in my case, and I hope it will help you.
M.J. (Maarten) Pols
Products and Services
System and application administrator
[cid:image002.jpg@01DAD1DB.05CFD300]
Botter 11-29, 8232 JN Lelystad, The Netherlands (also postal address)
Berkenweg 7, Amersfoort | Informaticalaan 8, Delft
Telephone +31 (0)320 294292
Internet www.hkv.nl/en/
HKV, knowledge entrepreneurs in flood risk and water resources management
Van: thredds <thredds-bounces@xxxxxxxxxxxxxxxx> Namens Jim Fluke
Verzonden: Tuesday, 9 July 2024 00:04
Aan: thredds@xxxxxxxxxxxxxxxx
Onderwerp: [thredds] Authentication problems with the TDS and pydap
## Let op: deze mail is afkomstig van een externe afzender. Meer informatie
over waarom dit belangrijk is<https://aka.ms/LearnAboutSenderIdentification>
Hello,
I'm now trying to get user authentication working with our thredds-docker based
TDS. I'm pretty sure I have the configuration set up to enable authentication
as described in the TDS manual's "Restrict Access To The
TDS<https://docs.unidata.ucar.edu/tds/current/userguide/restict_access_to_tds.html#restrict-access-by-dataset-in-tds-catalogs>"
page. And I have verified this by accessing the TDS from a browser and having
the credentials entry pop-up window display and work correctly.
But, I can't get the authentication to work in Python with pydap. According to
the pydap documentation the credentials should be added to the URL this way:
>>> from pydap.client import open_url
>>> dataset =
>>> open_url('http://username:password@xxxxxxxxxxxxxxxxxx/path/to/dataset')
But because Digested
Passwords<https://docs.unidata.ucar.edu/tds/current/userguide/digested_passwords.html>
are enabled for our TDS, it seems clear that I should use the digested
password, so this is what I tried:
>>> from pydap.client import open_url
>>> dataset =
>>> open_url('http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b
2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf<mailto:2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf>
')
But it does not work. Here is the output:
@ ~/devRepos/thredds-dpc-gh-actual/tests$ docker-compose run --rm test_opendap
url:
http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b
2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf<mailto:2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf>
Traceback (most recent call last):
File "/app/opendap_pydap.py", line 8, in <module>
dataset = open_url(url)
^^^^^^^^^^^^^
File "/opt/conda/lib/python3.12/site-packages/pydap/client.py", line 68, in
open_url
handler = pydap.handlers.dap.DAPHandler(url, application, session,
output_grid,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py", line 71,
in __init__
self.make_dataset()
File "/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py", line 96,
in make_dataset
self.dataset_from_dap2()
File "/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py", line
109, in dataset_from_dap2
pydap.net.raise_for_status(r)
File "/opt/conda/lib/python3.12/site-packages/pydap/net.py", line 38, in
raise_for_status
raise HTTPError(
webob.exc.HTTPError: 401 Unauthorized
<!doctype html><html lang="en"><head><title>HTTP Status 401 –
Unauthorized</title><style type="text/css">body
{font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-co
lor:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p
{font-size:12px;} a {color:black;} .line
{height:1px;background-color:#525D76;border:none;}</style></head><bod
y><h1>HTTP Status 401 – Unauthorized</h1><hr class="line" /><p><b>Type</b>
Status Report</p><p><b>Description</b> The request has not been applied to the
target resource because it lacks va
lid authentication credentials for that resource.</p><hr class="line"
/><h3>Apache Tomcat</h3></body></html>
So, am I right to be using the digested password? Do you see anything else that
could be wrong? Why does this work for the browser but not for pydap?
I will add that the algorithm for the CredentialHandler is "sha-512" in the
~tomcat/conf/server.xml file inside the container, so that is why the digested
password is an sha512 digest. And the clear text password is "flukeTmp". I'll
be changing that for our production system.
And, all of this - the TDS configuration and the test python script with the
above URL - are now checked in to our
thredds-dpc<https://github.com/JimFluke/thredds-dpc/tree/master> repository on
GitHub so you can look at the details there.
Any help would be greatly appreciated.
Thanks,
Jim
