oh,
here is a script with pyDAP and netCDF4:
# Example of failed requet from a HYRAX server
url = "
http://ingria.coas.oregonstate.edu:80/opendap/aggregation/ocean_time_aggregation.ncml
"
# pydap works fine:
from pydap.client import open_url
dataset = open_url(url)
u = dataset['u']
print u
print "shape is:", u.shape
u_data = u[0, 0, 0:10, 0:10] # this does not work--"RuntimeError: NetCDF:
file not found"
print "pydap did fine"
# but netCDF4 fails
from netCDF4 import Dataset
nc = Dataset('[log][show=fetch]' + url)
print("Dataset loaded OK")
u = nc.variables['u']
print("Successfully Extracted a varaible object:")
print(u)
u_shape = u.shape # this works ok
print("The variable object is seemingly valid:")
print("the shape is %s" % (u_shape,))
print("but I can't extract the actual data")
u_data = u[0, 0, 0:10, 0:10] # this does not work--"RuntimeError: NetCDF:
file not found"
On Mon, Oct 17, 2016 at 3:54 PM, Chris Barker <chris.barker@xxxxxxxx> wrote:
> On Mon, Oct 17, 2016 at 3:33 PM, dmh@xxxxxxxx <dmh@xxxxxxxx> wrote:
>
>> The difference with pydap is probably that you adding a constraint
>> that is causing the server to ignore whatever variable
>> is causing the problem.
>
>
> I don't think so -- the request is made almost identically -- see the
> enclosed script, that has both teh pydap code and the neCDF4 code.
>
>
>> If you ask pydap for the whole
>> dataset, does it fail?
>> (e.g. something like:
>> http://ingria.coas.oregonstate.edu/opendap/aggregation/ocean
>> _time_aggregation.ncml.ascii
>>
>
> I dopnt hinkt you can point pydap at a .ascii url, can you? It needs to
> speak OpenDAP...
>
> if I try to access the entire variable, it crashes, 'cause it is way too
> big:
>
> shape is: (1026, 40, 522, 309)
> That's 24 GB of data.
>
> when I open the dataset, I get all the metadata fine -- it knows all the
> variable, there size, etc -- both with pydap and netCDF4
>
> -CHB
>
>
> =Dennis Heimbigner
>> Unidata
>>
>> On 10/17/2016 4:16 PM, Chris Barker wrote:
>>
>>> On Mon, Oct 17, 2016 at 2:36 PM, dmh@xxxxxxxx <mailto:dmh@xxxxxxxx>
>>> <dmh@xxxxxxxx <mailto:dmh@xxxxxxxx>> wrote:
>>>
>>> The log and show=fetch appear to not work under python
>>> for some reason; stdout/stderr redirect?
>>>
>>>
>>> I don't think so -- and I do get a lot of "Log:prefetch" messages, so
>>> stdout.stderr seems to be working.
>>>
>>>
>>>
>>>
>>> In any case I can duplicate the error using wget e.g.
>>> wget
>>> 'http://ingria.coas.oregonstate.edu:80/opendap/aggregation/o
>>> cean_time_aggregation.ncml.dods
>>> <http://ingria.coas.oregonstate.edu:80/opendap/aggregation/o
>>> cean_time_aggregation.ncml.dods>'
>>>
>>>
>>>
>>> hmm, I don't seem to have wget, but with curl, I get the below results,
>>> ending in:
>>>
>>> Data:
>>>
>>> Error {
>>>
>>> code = -111;
>>>
>>> message = "libdap error transmitting DataDDS: Could not read the
>>> variable `ntimes'.";
>>>
>>> -- if that's the fatal error, then yup -- not the netcdf lib -- and yet,
>>> we can get the data with PyDAP, and if I put this:
>>>
>>>
>>> http://ingria.coas.oregonstate.edu/opendap/aggregation/ocean
>>> _time_aggregation.ncml.ascii?u[0][0][0:10][0:10]
>>>
>>> into my browser, I do get data.
>>>
>>> I have asked this on support@xxxxxxxxxxx <mailto:support@xxxxxxxxxxx> --
>>>
>>> and they have pointed out that that is a pretty old version of Hyrax --
>>> _maybe_ that's the problem.
>>>
>>> -CHB
>>>
>>>
>>> $ curl
>>> 'http://ingria.coas.oregonstate.edu:80/opendap/aggregation/o
>>> cean_time_aggregation.ncml.dods'
>>>
>>> Dataset {
>>>
>>> Int32 ntimes;
>>>
>>> Int32 ndtfast;
>>>
>>> Float64 dt;
>>>
>>> Float64 dtfast;
>>>
>>> Float64 dstart;
>>>
>>> Int32 nHIS;
>>>
>>> Int32 ndefHIS;
>>>
>>> Int32 nRST;
>>>
>>> Int32 ntsAVG;
>>>
>>> Int32 nAVG;
>>>
>>> Int32 ndefAVG;
>>>
>>> Float64 Falpha;
>>>
>>> Float64 Fbeta;
>>>
>>> Float64 Fgamma;
>>>
>>> Float64 nl_tnu2[tracer = 2];
>>>
>>> Float64 nl_visc2;
>>>
>>> Float64 Akt_bak[tracer = 2];
>>>
>>> Float64 Akv_bak;
>>>
>>> Float64 Akk_bak;
>>>
>>> Float64 Akp_bak;
>>>
>>> Float64 rdrg;
>>>
>>> Float64 rdrg2;
>>>
>>> Float64 Zob;
>>>
>>> Float64 Zos;
>>>
>>> Float64 Znudg;
>>>
>>> Float64 M2nudg;
>>>
>>> Float64 M3nudg;
>>>
>>> Float64 Tnudg[tracer = 2];
>>>
>>> Float64 FSobc_in[boundary = 4];
>>>
>>> Float64 FSobc_out[boundary = 4];
>>>
>>> Float64 M2obc_in[boundary = 4];
>>>
>>> Float64 M2obc_out[boundary = 4];
>>>
>>> Float64 Tobc_in[boundary = 4][tracer = 2];
>>>
>>> Float64 Tobc_out[boundary = 4][tracer = 2];
>>>
>>> Float64 M3obc_in[boundary = 4];
>>>
>>> Float64 M3obc_out[boundary = 4];
>>>
>>> Float64 rho0;
>>>
>>> Float64 gamma2;
>>>
>>> Int32 LtracerSrc[tracer = 2];
>>>
>>> Int32 spherical;
>>>
>>> Float64 xl;
>>>
>>> Float64 el;
>>>
>>> Int32 Vtransform;
>>>
>>> Int32 Vstretching;
>>>
>>> Float64 theta_s;
>>>
>>> Float64 theta_b;
>>>
>>> Float64 Tcline;
>>>
>>> Float64 hc;
>>>
>>> Float64 s_rho[s_rho = 40];
>>>
>>> Float64 s_w[s_w = 41];
>>>
>>> Grid {
>>>
>>> Array:
>>>
>>> Float64 Cs_r[s_rho = 40];
>>>
>>> Maps:
>>>
>>> Float64 s_rho[s_rho = 40];
>>>
>>> } Cs_r;
>>>
>>> Grid {
>>>
>>> Array:
>>>
>>> Float64 Cs_w[s_w = 41];
>>>
>>> Maps:
>>>
>>> Float64 s_w[s_w = 41];
>>>
>>> } Cs_w;
>>>
>>> Float64 user[Nuser = 25];
>>>
>>> Float64 h[eta_rho = 522][xi_rho = 310];
>>>
>>> Float64 f[eta_rho = 522][xi_rho = 310];
>>>
>>> Float64 pm[eta_rho = 522][xi_rho = 310];
>>>
>>> Float64 pn[eta_rho = 522][xi_rho = 310];
>>>
>>> Float64 lon_rho[eta_rho = 522][xi_rho = 310];
>>>
>>> Float64 lat_rho[eta_rho = 522][xi_rho = 310];
>>>
>>> Float64 lon_u[eta_u = 522][xi_u = 309];
>>>
>>> Float64 lat_u[eta_u = 522][xi_u = 309];
>>>
>>> Float64 lon_v[eta_v = 521][xi_v = 310];
>>>
>>> Float64 lat_v[eta_v = 521][xi_v = 310];
>>>
>>> Float64 lon_psi[eta_psi = 521][xi_psi = 309];
>>>
>>> Float64 lat_psi[eta_psi = 521][xi_psi = 309];
>>>
>>> Float64 angle[eta_rho = 522][xi_rho = 310];
>>>
>>> Float64 mask_rho[eta_rho = 522][xi_rho = 310];
>>>
>>> Float64 mask_u[eta_u = 522][xi_u = 309];
>>>
>>> Float64 mask_v[eta_v = 521][xi_v = 310];
>>>
>>> Float64 mask_psi[eta_psi = 521][xi_psi = 309];
>>>
>>> Float64 ocean_time[ocean_time = 1026];
>>>
>>> Float32 zeta[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
>>>
>>> Float32 ubar[ocean_time = 1026][eta_u = 522][xi_u = 309];
>>>
>>> Float32 vbar[ocean_time = 1026][eta_v = 521][xi_v = 310];
>>>
>>> Float32 u[ocean_time = 1026][s_rho = 40][eta_u = 522][xi_u = 309];
>>>
>>> Float32 v[ocean_time = 1026][s_rho = 40][eta_v = 521][xi_v = 310];
>>>
>>> Float32 w[ocean_time = 1026][s_w = 41][eta_rho = 522][xi_rho = 310];
>>>
>>> Float32 temp[ocean_time = 1026][s_rho = 40][eta_rho = 522][xi_rho =
>>> 310];
>>>
>>> Float32 salt[ocean_time = 1026][s_rho = 40][eta_rho = 522][xi_rho =
>>> 310];
>>>
>>> Float32 AKv[ocean_time = 1026][s_w = 41][eta_rho = 522][xi_rho =
>>> 310];
>>>
>>> Float32 AKt[ocean_time = 1026][s_w = 41][eta_rho = 522][xi_rho =
>>> 310];
>>>
>>> Float32 AKs[ocean_time = 1026][s_w = 41][eta_rho = 522][xi_rho =
>>> 310];
>>>
>>> Float32 tke[ocean_time = 1026][s_w = 41][eta_rho = 522][xi_rho =
>>> 310];
>>>
>>> Float32 shflux[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
>>>
>>> Float32 latent[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
>>>
>>> Float32 sensible[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
>>>
>>> Float32 lwrad[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
>>>
>>> Float32 swrad[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
>>>
>>> } ocean_time_aggregation.ncml;
>>>
>>> Data:
>>>
>>> Error {
>>>
>>> code = -111;
>>>
>>> message = "libdap error transmitting DataDDS: Could not read the
>>> variable `ntimes'.";
>>>
>>> };
>>>
>>>
>>> --
>>>
>>> Christopher Barker, Ph.D.
>>> Oceanographer
>>>
>>> Emergency Response Division
>>> NOAA/NOS/OR&R (206) 526-6959 voice
>>> 7600 Sand Point Way NE (206) 526-6329 fax
>>> Seattle, WA 98115 (206) 526-6317 main reception
>>>
>>> Chris.Barker@xxxxxxxx <mailto:Chris.Barker@xxxxxxxx>
>>>
>>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris.Barker@xxxxxxxx
>
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@xxxxxxxx
