[thredds] Authentication problems with the TDS and pydap

  • To: thredds@xxxxxxxxxxxxxxxx
  • Subject: [thredds] Authentication problems with the TDS and pydap
  • From: Jim Fluke <james.fluke@xxxxxxxxxxxxx>
  • Date: Mon, 8 Jul 2024 16:04:18 -0600
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=colostate.edu; dmarc=pass action=none header.from=colostate.edu; dkim=pass header.d=colostate.edu; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rcOozxEStBSzksIBVF2vK+PlSIU48zJzdnRfjd7AHdo=; b=Fm9HQF4ZQC63SvdKp8+nOOd3Pc+sc9ZvTltCkP+iu6U9KXdruaWP83emGfwp05YqZKJMXahcP6rXlqXos6Rb2HkZgx1F0+3KCcMyl5jeb4nFnIfTAPiU1L2upjPQRKSdiYyPndHRGPlX2lFMBk0TWDh9UlxcNCO2tQt0VR9Az24pX4bxTXf5zGTdn3yHx7bJfsEMNCzmdg9tW39diGUKmUes9qhWYhxQrRkJqXqZQkTFBQPDQ51nP+rLrn+z/oe1wwQE5pDtRDZh8i0G5xKFzbOjMNhQIQ95f+dOrn+eV+jYd9p8r74BlhBdQKZTFCZduG9gWPKDUQtrs3R1BC8B2Q==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hyoOyrL4bi85ACQ9TyRp9yJyeaJy5lHrClViKsN5R7hpQER0DePDkRLtuZDRPI+aBbkwTKKbj2xzfMX4qeAgyCZxiubgQax+jP87Ml4zXpX4gvpTNLqa6EOPNpLEnqVkVap88B6j50/slYPTbjoKNnyWU2ScynI+Vrz9it2jYigX5vHfeCCnZEIHs/u0/0G39F3jdbm6XckUu8STEgYGcVijRAtQFHKgFM4YLd5bF6bw2taFSigui2i4C9GNoZhNmQ0kLbux0yZdM4iMmXw3UBl4NnDwx2EvZ51AgMxof7I0etfrIkr0/xadThHBHrUkLB+KgKPWU9IDOStvEPCmHA==
  • Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=colostate.edu;
Hello,

I'm now trying to get user authentication working with our thredds-docker based TDS. I'm pretty sure I have the configuration set up to enable authentication as described in the TDS manual's "Restrict Access To The TDS <https://docs.unidata.ucar.edu/tds/current/userguide/restict_access_to_tds.html#restrict-access-by-dataset-in-tds-catalogs>" page. And I have verified this by accessing the TDS from a browser and having the credentials entry pop-up window display and work correctly.

But, I can't get the authentication to work in Python with pydap. According to the pydap documentation the credentials should be added to the URL this way:

 from pydap.client import open_url
>>> dataset = open_url('http://username:password@xxxxxxxxxxxxxxxxxx/path/to/dataset')

But because Digested Passwords <https://docs.unidata.ucar.edu/tds/current/userguide/digested_passwords.html> are enabled for our TDS, it seems clear that I should use the digested password, so this is what I tried:

 from pydap.client import open_url
>>> dataset = open_url('http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b
2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf
')

But it does not work. Here is the output:

@ ~/devRepos/thredds-dpc-gh-actual/tests$ docker-compose run --rm test_opendap url: http://fluke:d1ef3ce7e7c41de74192a362524ad0a460692a222d9dd796ee383b56e446d749$1$d03ce0f88475505a68bd0eb37fa570df8120e59ccf62a4f580a55ad612f695c0e385893fe7205f7c181b221ab49bc817d4a33a2b 2bb727fdc0ee3420e7e5b99e@localhost:7000/thredds/dodsC/cloudsat-data/2B-GEOPROF.P1_R05/2008/366/2008366031107_14239_CS_2B-GEOPROF_GRANULE_P1_R05_E02_F00.hdf

Traceback (most recent call last):
 File "/app/opendap_pydap.py", line 8, in <module>
   dataset = open_url(url)
             ^^^^^^^^^^^^^
 File "/opt/conda/lib/python3.12/site-packages/pydap/client.py", line 68, in open_url    handler = pydap.handlers.dap.DAPHandler(url, application, session, output_grid,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py", line 71, in __init__
   self.make_dataset()
 File "/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py", line 96, in make_dataset
   self.dataset_from_dap2()
 File "/opt/conda/lib/python3.12/site-packages/pydap/handlers/dap.py", line 109, in dataset_from_dap2
   pydap.net.raise_for_status(r)
 File "/opt/conda/lib/python3.12/site-packages/pydap/net.py", line 38, in raise_for_status
   raise HTTPError(
webob.exc.HTTPError: 401 Unauthorized
<!doctype html><html lang="en"><head><title>HTTP Status 401 – Unauthorized</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-co lor:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><bod y><h1>HTTP Status 401 – Unauthorized</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Description</b> The request has not been applied to the target resource because it lacks va lid authentication credentials for that resource.</p><hr class="line" /><h3>Apache Tomcat</h3></body></html>

So, am I right to be using the digested password? Do you see anything else that could be wrong? Why does this work for the browser but not for pydap?

I will add that the algorithm for the CredentialHandler is "sha-512" in the ~tomcat/conf/server.xml file inside the container, so that is why the digested password is an sha512 digest. And the clear text password is "flukeTmp". I'll be changing that for our production system.

And, all of this - the TDS configuration and the test python script with the above URL - are now checked in to our thredds-dpc <https://github.com/JimFluke/thredds-dpc/tree/master> repository on GitHub so you can look at the details there.

Any help would be greatly appreciated.

Thanks,
Jim
  • 2024 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: