Re: [thredds] NetCDF Access failure with matlab and python

  • To: Dennis Heimbigner <dmh@xxxxxxxx>
  • Subject: Re: [thredds] NetCDF Access failure with matlab and python
  • From: Thomas Cook <tmcook@xxxxxxxx>
  • Date: Fri, 18 Jan 2019 11:08:04 -0800
Dennis,
Thanks for your help.
After thinking about it, I realized that the log I sent was from the
non-working version, and was likely blocked by TDS. I went back with a
working version and the request is shown as
2019-01-18T19:05:07.310 +0000 [    827256][      20] INFO  -
threddsServlet - Remote host: 172.17.0.1 - Request: "GET
/thredds/dodsC/HFR/USWC/6km/hourly/RTV/HFRADAR_US_West_Coast_6km_Resolution_Hourly_RTV_best.ncd.dods?v.v[63939][0:366][0:233]
HTTP/1.0"

So there are [ and ] in the request.
Tom

On Fri, Jan 18, 2019 at 10:33 AM Dennis Heimbigner <dmh@xxxxxxxx> wrote:
>
> This one example does not include any of the now prohibited
> characters (e.g. '[' and ']'), so I do not believe that encoding
> is the problem.
>
> On 1/16/2019 9:40 PM, Thomas Cook wrote:
> > I do not know how to debug the matlab or python clients, but this is
> > what shows up in the threddsServlet.log
> >
> > Request: "GET 
> > /thredds/dodsC/HFR/USWC/6km/hourly/RTV/HFRADAR_US_West_Coast_6km_Resolution_Hourly_RTV_best.ncd.dods?lat,lon,site%5flat,site%5flon,site%5fcode,site%5fnetCode,procParams
> > HTTP/1.0"
> >
> > Thanks
> > Tom
> >
> > On Wed, Jan 16, 2019 at 7:47 PM Dennis Heimbigner <dmh@xxxxxxxx> wrote:
> >> Is there anyway you can see the exact URL being sent to the server?
> >>
> >> On 1/16/2019 8:24 PM, Thomas Cook wrote:
> >>> Thanks Sean.
> >>> The server is http://hfrnet-tds.ucsd.edu/thredds (although I just
> >>> downgraded back to 4.6.12-SNAPSHOT). I will see about making my test
> >>> server accessible to you, but that will take a little time to
> >>> configure on my end.
> >>> Let me know if you think I should open an issue on the github page and
> >>> move the discussion there.
> >>> Tom
> >>>
> >>> On Wed, Jan 16, 2019 at 6:32 PM Sean Arms <sarms@xxxxxxxx> wrote:
> >>>> Hi Tom,
> >>>>
> >>>> So you are hitting up an opendap endpoint using all three libraries / 
> >>>> tools (python, matlab, and panoply)? Is this particular server publicly 
> >>>> accessible?
> >>>>
> >>>> I have a bad feeling that this is due to URL encoding (or lack thereof) 
> >>>> on the client side and changes made in tomcat 8.5, which it looks like 
> >>>> the thredds-docker repository bumped up to with the 4.6.12 release.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Sean
> >>>>
> >>>>
> >>>> On Wed, Jan 16, 2019 at 7:06 PM Thomas Cook <tmcook@xxxxxxxx> wrote:
> >>>>> HI I just wanted to add some details.
> >>>>>
> >>>>> The functionality works when I downgraded to the docker container
> >>>>> 4.6.12-SNAPSHOT
> >>>>>
> >>>>> My data is part of a FMRC collection.
> >>>>>
> >>>>> Thanks,
> >>>>> Tom
> >>>>>
> >>>>> On Wed, Jan 16, 2019 at 4:51 PM Thomas Cook <tmcook@xxxxxxxx> wrote:
> >>>>>> HI
> >>>>>> I just yesterday updated my thredds-docker to 4.6.12 from
> >>>>>> 4.6.12-snapshot. It seems to be running fine, except I'm getting
> >>>>>> errors using matlab ncread and python netcdf4.dataset. When I try to
> >>>>>> access any of the multi-dimensional (time, lat, lon) variables I get
> >>>>>> the following errors
> >>>>>> with matlab:
> >>>>>> Error using netcdflib
> >>>>>> The NetCDF library encountered an error during execution of
> >>>>>> 'getVarsShort' function - 'Access failure (-77)'.
> >>>>>>
> >>>>>> Error in netcdf.getVar (line 136)
> >>>>>>       data = netcdflib(funcstr,ncid,varid,varargin{:});
> >>>>>>
> >>>>>> Error in internal.matlab.imagesci.nc/read (line 635)
> >>>>>>                   data  = netcdf.getVar(gid, varid, ...
> >>>>>>
> >>>>>> Error in ncread (line 58)
> >>>>>> vardata = ncObj.read(varName, varargin{:});
> >>>>>>
> >>>>>> Error in readUVfromTDS (line 56)
> >>>>>> u=ncread(url, 'u_mean', [1 1 1], [2 2 2]);
> >>>>>>
> >>>>>>
> >>>>>> and with python:
> >>>>>> netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.__getitem__()
> >>>>>>
> >>>>>> netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable._get()
> >>>>>>
> >>>>>> netCDF4/_netCDF4.pyx in netCDF4._netCDF4._ensure_nc_success()
> >>>>>>
> >>>>>> RuntimeError: NetCDF: Access failure
> >>>>>>
> >>>>>> I know that the latest docker contains netcdf 4.6, where as the
> >>>>>> snapshot had 4.4. Could it be a version problem with the netCD4 python
> >>>>>> and matlab libraries? I should be running the latest netcdf python
> >>>>>> version and not too far behind with matlab versions.
> >>>>>>
> >>>>>> The weird things are that the index variables are accessible (time,
> >>>>>> lat, lon etc),  and the multiple-dimension data is accessible just
> >>>>>> fine using the OpenDap web interface and Panoply. So I'm a little
> >>>>>> confused. I'm addressing this list since I think there might be some
> >>>>>> matlab and python expertise here and I think there were some messages
> >>>>>> about the Access failure error in the past.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Tom
> >>>>> _______________________________________________
> >>>>> NOTE: All exchanges posted to Unidata maintained email lists are
> >>>>> recorded in the Unidata inquiry tracking system and made publicly
> >>>>> available through the web.  Users who post to any of the lists we
> >>>>> maintain are reminded to remove any personal information that they
> >>>>> do not want to be made public.
> >>>>>
> >>>>>
> >>>>> thredds mailing list
> >>>>> thredds@xxxxxxxxxxxxxxxx
> >>>>> For list information or to unsubscribe,  visit: 
> >>>>> http://www.unidata.ucar.edu/mailing_lists/
> >>> _______________________________________________
> >>> NOTE: All exchanges posted to Unidata maintained email lists are
> >>> recorded in the Unidata inquiry tracking system and made publicly
> >>> available through the web.  Users who post to any of the lists we
> >>> maintain are reminded to remove any personal information that they
> >>> do not want to be made public.
> >>>
> >>>
> >>> thredds mailing list
> >>> thredds@xxxxxxxxxxxxxxxx
> >>> For list information or to unsubscribe,  visit: 
> >>> http://www.unidata.ucar.edu/mailing_lists/
> >> _______________________________________________
> >> NOTE: All exchanges posted to Unidata maintained email lists are
> >> recorded in the Unidata inquiry tracking system and made publicly
> >> available through the web.  Users who post to any of the lists we
> >> maintain are reminded to remove any personal information that they
> >> do not want to be made public.
> >>
> >>
> >> thredds mailing list
> >> thredds@xxxxxxxxxxxxxxxx
> >> For list information or to unsubscribe,  visit: 
> >> http://www.unidata.ucar.edu/mailing_lists/
>



  • 2019 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: