Just to clarify, I made references to potential misconfigurations on the
hyrax server only b/c I saw statements to that effect in the previous
emails on this thread. I'm not in a position to make assessments myself.
And as I mentioned, NANOOS is also not in a real position to offer
significant changes to that hyrax server.
Any recommendations to upgrade to a more recent version should be made
directly to the server maintainer; I don't remember her name, but the model
PI is Alexandre Kurapov, kurapov@xxxxxxxxxxxxxxxxxxxx. Please cc me.
Also, Chris Barker replied separately saying that their needs our almost
always met by THREDDS servers, so he thought our operational and actively
maintained NANOOS THREDDS server may meet his needs. If the hyrax server in
question was in fact originally deployed only to meet NOAA ORR needs, and
those needs can now be met by our THREDDS server, the decision might be to
discontinue the hyrax server. But that'll be up to Alexandre.
Cheers,
-Emilio
On Wed, Oct 19, 2016 at 5:34 AM, Nathan Potter <ndp@xxxxxxxxxxx> wrote:
>
> I would appreciate knowing what part of the behavior/output of the
> ingria.coas.oregonstate.edu Hyrax server has led to the comments that it
> is not configured correctly.
>
> - Is it simply that the server is producing an error when trying to
> transmit the “ntimes” variable?
> - Is there something specific in the NcML content that speaks directly to
> this?
> - In the context of this thread does “configuration problem/issue” mean an
> improperly constructed NcML file?
>
> Can anyone provide some insight?
>
> By asking these questions I am trying to understand, and not in anyway
> dispute, the assertion that the server is “misconfigured”. I just want to
> tease apart the possibility that we are seeing a fundamental bug problem
> from a configuration issue.
>
> I do know that at NASA’s request and expense a significant number of
> changes and improvements have been made to Hyrax’s NcML features, but this
> work happened long after the release of version 1.9.0.
>
> In light of this I think the first step should be to instigate upgrading
> the server, as version 1.9.0 is a long way back and not only does the
> latest release (1.13.1) offer an improved NcML implementation it also
> contains numerous security updates.
>
> Regardless, any clarification that could be offered w.r.t. the
> mis-configuration assertion would be very helpful to me.
>
>
> Sincerely,
>
> Nathan
>
>
>
>
> > On Oct 18, 2016, at 11:30 PM, Emilio Mayorga <emiliomayorga@xxxxxxxxx>
> wrote:
> >
> > I'm afraid I don't have much helpful insight to bring into this. The
> model behind that Hyrax server is indeed a NANOOS model, but the hyrax
> server itself was not set up and is not maintained by one of us in the
> NANOOS DMAC team. In fact, if I remember correctly the server is somewhat
> duplicative of a THREDDS server we do maintain for the same model output;
> it was setup about two years ago, I think to support an application at NOAA
> ORR that took advantage of some specific hyrax capability missing in
> THREDDS' OPeNDAP (Chris Barker would know better than me), partly through
> help and persuasion from Rich Signell; again, sketchy memory here. I was
> never thrilled with the idea of having a hyrax server in the mix when none
> of us had the capacity to invest in learning how to properly maintain it,
> and already had a THREDDS server for the same model. I'm not surprised to
> hear it's an old version, too.
> >
> > Sorry ...
> >
> > -Emilio
> >
> >
> > On Mon, Oct 17, 2016 at 5:36 PM, Chris Barker <chris.barker@xxxxxxxx>
> wrote:
> > Folks,
> >
> > I manged to do a tcpdump, and can (sortof) see what's going on -- though
> I can't really make sense of it -- maybe one of you can?
> >
> > This is running the enclosed script, to it first does the successful
> pydap request, then the unsuccessful netCDF4 request.
> >
> > (though it looking like this is a poorly configured server...)
> >
> > -CHB
> >
> >
> > On Mon, Oct 17, 2016 at 4:49 PM, Nathan Potter <ndp@xxxxxxxxxxx> wrote:
> > By which I meant - just deference the URL:
> >
> > http://ingria.coas.oregonstate.edu/opendap/aggregation/ocean_time_
> aggregation.ncml
> >
> > To get the NcML file:
> >
> > curl http://ingria.coas.oregonstate.edu/opendap/
> aggregation/ocean_time_aggregation.ncml
> >
> >
> >
> > Nathan
> >
> >
> >
> > > On Oct 17, 2016, at 4:37 PM, Nathan Potter <ndp@xxxxxxxxxxx> wrote:
> > >
> > > Hi,
> > >
> > > Seems like no one has tried to actually ask for the NcML file:
> > >
> > > http://ingria.coas.oregonstate.edu/opendap/aggregation/ocean_time_
> aggregation.ncml
> > >
> > >
> > > N
> > >
> > >
> > >> On Oct 17, 2016, at 4:35 PM, Roy Mendelssohn - NOAA Federal <
> roy.mendelssohn@xxxxxxxx> wrote:
> > >>
> > >> Hi Emilio:
> > >>
> > >> I believe this is a NaNOOS server. If you can read over the thread,,
> any insight would be appreciated, What is in the .ncml file could be of
> interest.
> > >>
> > >> Thanks,
> > >>
> > >> -roy
> > >>
> > >>> On Oct 17, 2016, at 4:20 PM, Chris Barker <Chris.Barker@xxxxxxxx>
> wrote:
> > >>>
> > >>> On Mon, Oct 17, 2016 at 3:57 PM, Roy Mendelssohn - NOAA Federal <
> roy.mendelssohn@xxxxxxxx> wrote:
> > >>> I find that if you go to the html page, many of the variables at
> the top can't be accessed requesting ascii, so there are clearly problems
> with how this is set up
> > >>>
> > >>> that does point to server issues.
> > >>>
> > >>> I am wondering if these are defined in the .ncml. It would be
> interesting to see what the .ncml file looks like.
> > >>>
> > >>> One thing to consider also is pydap does not call the C library
> IIRC, it is a pure python implementation
> > >>>
> > >>> Exactly -- so it takes the netcdf C lib out of the picture -- which
> means there is a protocol that pydap and hyrax are speaking that lets the
> data be accessed...
> > >>>
> > >>> If anyone knows how to see what actual requests are being made by
> the two, we might be able to figure out where the failure is...
> > >>>
> > >>> -CHB
> > >>>
> > >>>
> > >>> -Roy
> > >>>
> > >>>> On Oct 17, 2016, at 3:33 PM, dmh@xxxxxxxx wrote:
> > >>>>
> > >>>> The difference with pydap is probably that you adding a constraint
> > >>>> that is causing the server to ignore whatever variable
> > >>>> is causing the problem. If you ask pydap for the whole
> > >>>> dataset, does it fail?
> > >>>> (e.g. something like:
> > >>>> http://ingria.coas.oregonstate.edu/opendap/aggregation/ocean_time_
> aggregation.ncml.ascii
> > >>>> =Dennis Heimbigner
> > >>>> Unidata
> > >>>>
> > >>>> On 10/17/2016 4:16 PM, Chris Barker wrote:
> > >>>>> On Mon, Oct 17, 2016 at 2:36 PM, dmh@xxxxxxxx <mailto:dmh@xxxxxxxx
> >
> > >>>>> <dmh@xxxxxxxx <mailto:dmh@xxxxxxxx>> wrote:
> > >>>>>
> > >>>>> The log and show=fetch appear to not work under python
> > >>>>> for some reason; stdout/stderr redirect?
> > >>>>>
> > >>>>>
> > >>>>> I don't think so -- and I do get a lot of "Log:prefetch" messages,
> so
> > >>>>> stdout.stderr seems to be working.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> In any case I can duplicate the error using wget e.g.
> > >>>>> wget
> > >>>>> 'http://ingria.coas.oregonstate.edu:80/opendap/
> aggregation/ocean_time_aggregation.ncml.dods
> > >>>>> <http://ingria.coas.oregonstate.edu:80/opendap/
> aggregation/ocean_time_aggregation.ncml.dods>'
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> hmm, I don't seem to have wget, but with curl, I get the below
> results,
> > >>>>> ending in:
> > >>>>>
> > >>>>> Data:
> > >>>>>
> > >>>>> Error {
> > >>>>>
> > >>>>> code = -111;
> > >>>>>
> > >>>>> message = "libdap error transmitting DataDDS: Could not read the
> > >>>>> variable `ntimes'.";
> > >>>>>
> > >>>>> -- if that's the fatal error, then yup -- not the netcdf lib --
> and yet,
> > >>>>> we can get the data with PyDAP, and if I put this:
> > >>>>>
> > >>>>>
> > >>>>> http://ingria.coas.oregonstate.edu/opendap/aggregation/ocean_time_
> aggregation.ncml.ascii?u[0][0][0:10][0:10]
> > >>>>>
> > >>>>> into my browser, I do get data.
> > >>>>>
> > >>>>> I have asked this on support@xxxxxxxxxxx <mailto:
> support@xxxxxxxxxxx> --
> > >>>>> and they have pointed out that that is a pretty old version of
> Hyrax --
> > >>>>> _maybe_ that's the problem.
> > >>>>>
> > >>>>> -CHB
> > >>>>>
> > >>>>>
> > >>>>> $ curl
> > >>>>> 'http://ingria.coas.oregonstate.edu:80/opendap/
> aggregation/ocean_time_aggregation.ncml.dods'
> > >>>>>
> > >>>>> Dataset {
> > >>>>>
> > >>>>> Int32 ntimes;
> > >>>>>
> > >>>>> Int32 ndtfast;
> > >>>>>
> > >>>>> Float64 dt;
> > >>>>>
> > >>>>> Float64 dtfast;
> > >>>>>
> > >>>>> Float64 dstart;
> > >>>>>
> > >>>>> Int32 nHIS;
> > >>>>>
> > >>>>> Int32 ndefHIS;
> > >>>>>
> > >>>>> Int32 nRST;
> > >>>>>
> > >>>>> Int32 ntsAVG;
> > >>>>>
> > >>>>> Int32 nAVG;
> > >>>>>
> > >>>>> Int32 ndefAVG;
> > >>>>>
> > >>>>> Float64 Falpha;
> > >>>>>
> > >>>>> Float64 Fbeta;
> > >>>>>
> > >>>>> Float64 Fgamma;
> > >>>>>
> > >>>>> Float64 nl_tnu2[tracer = 2];
> > >>>>>
> > >>>>> Float64 nl_visc2;
> > >>>>>
> > >>>>> Float64 Akt_bak[tracer = 2];
> > >>>>>
> > >>>>> Float64 Akv_bak;
> > >>>>>
> > >>>>> Float64 Akk_bak;
> > >>>>>
> > >>>>> Float64 Akp_bak;
> > >>>>>
> > >>>>> Float64 rdrg;
> > >>>>>
> > >>>>> Float64 rdrg2;
> > >>>>>
> > >>>>> Float64 Zob;
> > >>>>>
> > >>>>> Float64 Zos;
> > >>>>>
> > >>>>> Float64 Znudg;
> > >>>>>
> > >>>>> Float64 M2nudg;
> > >>>>>
> > >>>>> Float64 M3nudg;
> > >>>>>
> > >>>>> Float64 Tnudg[tracer = 2];
> > >>>>>
> > >>>>> Float64 FSobc_in[boundary = 4];
> > >>>>>
> > >>>>> Float64 FSobc_out[boundary = 4];
> > >>>>>
> > >>>>> Float64 M2obc_in[boundary = 4];
> > >>>>>
> > >>>>> Float64 M2obc_out[boundary = 4];
> > >>>>>
> > >>>>> Float64 Tobc_in[boundary = 4][tracer = 2];
> > >>>>>
> > >>>>> Float64 Tobc_out[boundary = 4][tracer = 2];
> > >>>>>
> > >>>>> Float64 M3obc_in[boundary = 4];
> > >>>>>
> > >>>>> Float64 M3obc_out[boundary = 4];
> > >>>>>
> > >>>>> Float64 rho0;
> > >>>>>
> > >>>>> Float64 gamma2;
> > >>>>>
> > >>>>> Int32 LtracerSrc[tracer = 2];
> > >>>>>
> > >>>>> Int32 spherical;
> > >>>>>
> > >>>>> Float64 xl;
> > >>>>>
> > >>>>> Float64 el;
> > >>>>>
> > >>>>> Int32 Vtransform;
> > >>>>>
> > >>>>> Int32 Vstretching;
> > >>>>>
> > >>>>> Float64 theta_s;
> > >>>>>
> > >>>>> Float64 theta_b;
> > >>>>>
> > >>>>> Float64 Tcline;
> > >>>>>
> > >>>>> Float64 hc;
> > >>>>>
> > >>>>> Float64 s_rho[s_rho = 40];
> > >>>>>
> > >>>>> Float64 s_w[s_w = 41];
> > >>>>>
> > >>>>> Grid {
> > >>>>>
> > >>>>> Array:
> > >>>>>
> > >>>>> Float64 Cs_r[s_rho = 40];
> > >>>>>
> > >>>>> Maps:
> > >>>>>
> > >>>>> Float64 s_rho[s_rho = 40];
> > >>>>>
> > >>>>> } Cs_r;
> > >>>>>
> > >>>>> Grid {
> > >>>>>
> > >>>>> Array:
> > >>>>>
> > >>>>> Float64 Cs_w[s_w = 41];
> > >>>>>
> > >>>>> Maps:
> > >>>>>
> > >>>>> Float64 s_w[s_w = 41];
> > >>>>>
> > >>>>> } Cs_w;
> > >>>>>
> > >>>>> Float64 user[Nuser = 25];
> > >>>>>
> > >>>>> Float64 h[eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float64 f[eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float64 pm[eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float64 pn[eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float64 lon_rho[eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float64 lat_rho[eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float64 lon_u[eta_u = 522][xi_u = 309];
> > >>>>>
> > >>>>> Float64 lat_u[eta_u = 522][xi_u = 309];
> > >>>>>
> > >>>>> Float64 lon_v[eta_v = 521][xi_v = 310];
> > >>>>>
> > >>>>> Float64 lat_v[eta_v = 521][xi_v = 310];
> > >>>>>
> > >>>>> Float64 lon_psi[eta_psi = 521][xi_psi = 309];
> > >>>>>
> > >>>>> Float64 lat_psi[eta_psi = 521][xi_psi = 309];
> > >>>>>
> > >>>>> Float64 angle[eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float64 mask_rho[eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float64 mask_u[eta_u = 522][xi_u = 309];
> > >>>>>
> > >>>>> Float64 mask_v[eta_v = 521][xi_v = 310];
> > >>>>>
> > >>>>> Float64 mask_psi[eta_psi = 521][xi_psi = 309];
> > >>>>>
> > >>>>> Float64 ocean_time[ocean_time = 1026];
> > >>>>>
> > >>>>> Float32 zeta[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float32 ubar[ocean_time = 1026][eta_u = 522][xi_u = 309];
> > >>>>>
> > >>>>> Float32 vbar[ocean_time = 1026][eta_v = 521][xi_v = 310];
> > >>>>>
> > >>>>> Float32 u[ocean_time = 1026][s_rho = 40][eta_u = 522][xi_u = 309];
> > >>>>>
> > >>>>> Float32 v[ocean_time = 1026][s_rho = 40][eta_v = 521][xi_v = 310];
> > >>>>>
> > >>>>> Float32 w[ocean_time = 1026][s_w = 41][eta_rho = 522][xi_rho =
> 310];
> > >>>>>
> > >>>>> Float32 temp[ocean_time = 1026][s_rho = 40][eta_rho = 522][xi_rho
> =
> > >>>>> 310];
> > >>>>>
> > >>>>> Float32 salt[ocean_time = 1026][s_rho = 40][eta_rho = 522][xi_rho
> =
> > >>>>> 310];
> > >>>>>
> > >>>>> Float32 AKv[ocean_time = 1026][s_w = 41][eta_rho = 522][xi_rho =
> 310];
> > >>>>>
> > >>>>> Float32 AKt[ocean_time = 1026][s_w = 41][eta_rho = 522][xi_rho =
> 310];
> > >>>>>
> > >>>>> Float32 AKs[ocean_time = 1026][s_w = 41][eta_rho = 522][xi_rho =
> 310];
> > >>>>>
> > >>>>> Float32 tke[ocean_time = 1026][s_w = 41][eta_rho = 522][xi_rho =
> 310];
> > >>>>>
> > >>>>> Float32 shflux[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float32 latent[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float32 sensible[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float32 lwrad[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> Float32 swrad[ocean_time = 1026][eta_rho = 522][xi_rho = 310];
> > >>>>>
> > >>>>> } ocean_time_aggregation.ncml;
> > >>>>>
> > >>>>> Data:
> > >>>>>
> > >>>>> Error {
> > >>>>>
> > >>>>> code = -111;
> > >>>>>
> > >>>>> message = "libdap error transmitting DataDDS: Could not read the
> > >>>>> variable `ntimes'.";
> > >>>>>
> > >>>>> };
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>>
> > >>>>> Christopher Barker, Ph.D.
> > >>>>> Oceanographer
> > >>>>>
> > >>>>> Emergency Response Division
> > >>>>> NOAA/NOS/OR&R (206) 526-6959 voice
> > >>>>> 7600 Sand Point Way NE (206) 526-6329 fax
> > >>>>> Seattle, WA 98115 (206) 526-6317 main reception
> > >>>>>
> > >>>>> Chris.Barker@xxxxxxxx <mailto:Chris.Barker@xxxxxxxx>
> > >>>>
> > >>>> _______________________________________________
> > >>>> NOTE: All exchanges posted to Unidata maintained email lists are
> > >>>> recorded in the Unidata inquiry tracking system and made publicly
> > >>>> available through the web. Users who post to any of the lists we
> > >>>> maintain are reminded to remove any personal information that they
> > >>>> do not want to be made public.
> > >>>>
> > >>>>
> > >>>> netcdfgroup mailing list
> > >>>> netcdfgroup@xxxxxxxxxxxxxxxx
> > >>>> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
> > >>>
> > >>> **********************
> > >>> "The contents of this message do not reflect any position of the
> U.S. Government or NOAA."
> > >>> **********************
> > >>> Roy Mendelssohn
> > >>> Supervisory Operations Research Analyst
> > >>> NOAA/NMFS
> > >>> Environmental Research Division
> > >>> Southwest Fisheries Science Center
> > >>> ***Note new address and phone***
> > >>> 110 Shaffer Road
> > >>> Santa Cruz, CA 95060
> > >>> Phone: (831)-420-3666
> > >>> Fax: (831) 420-3980
> > >>> e-mail: Roy.Mendelssohn@xxxxxxxx www: http://www.pfeg.noaa.gov/
> > >>>
> > >>> "Old age and treachery will overcome youth and skill."
> > >>> "From those who have been given much, much will be expected"
> > >>> "the arc of the moral universe is long, but it bends toward justice"
> -MLK Jr.
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>>
> > >>> Christopher Barker, Ph.D.
> > >>> Oceanographer
> > >>>
> > >>> Emergency Response Division
> > >>> NOAA/NOS/OR&R (206) 526-6959 voice
> > >>> 7600 Sand Point Way NE (206) 526-6329 fax
> > >>> Seattle, WA 98115 (206) 526-6317 main reception
> > >>>
> > >>> Chris.Barker@xxxxxxxx
> > >>
> > >> **********************
> > >> "The contents of this message do not reflect any position of the U.S .
> Government or NOAA."
> > >> **********************
> > >> Roy Mendelssohn
> > >> Supervisory Operations Research Analyst
> > >> NOAA/NMFS
> > >> Environmental Research Division
> > >> Southwest Fisheries Science Center
> > >> ***Note new address and phone***
> > >> 110 Shaffer Road
> > >> Santa Cruz, CA 95060
> > >> Phone: (831)-420-3666
> > >> Fax: (831) 420-3980
> > >> e-mail: Roy.Mendelssohn@xxxxxxxx www: http://www.pfeg.noaa.gov/
> > >>
> > >> "Old age and treachery will overcome youth and skill."
> > >> "From those who have been given much, much will be expected"
> > >> "the arc of the moral universe is long, but it bends toward justice"
> -MLK Jr.
> > >>
> > >
> > > = = =
> > > Nathan Potter ndp at opendap.org
> > > OPeNDAP, Inc. +1.541.231.3317
> > >
> >
> > = = =
> > Nathan Potter ndp at opendap.org
> > OPeNDAP, Inc. +1.541.231.3317
> >
> >
> >
> >
> > --
> >
> > Christopher Barker, Ph.D.
> > Oceanographer
> >
> > Emergency Response Division
> > NOAA/NOS/OR&R (206) 526-6959 voice
> > 7600 Sand Point Way NE (206) 526-6329 fax
> > Seattle, WA 98115 (206) 526-6317 main reception
> >
> > Chris.Barker@xxxxxxxx
> >
>
> = = =
> Nathan Potter ndp at opendap.org
> OPeNDAP, Inc. +1.541.231.3317
>
>
