It looks like conduit.ncep.noaa.gov is pulling data from both vm-lnx-conduit1
and vm-linux-conduit2 - conduit1 seems ok, it's just conduit2 that is showing
the large lags.
I don't know how things are set up exactly, but it might work to have
conduit.ncep.noaa.gov only request CONDUIT data from vm-lnx-conduit1 until the
problem with feeding from conduit2 is resolved?
Unidata folks, any suggestions from your end?
Thanks, we do appreciate all your work on our behalf!
Pete
<http://www.weather.com/tv/shows/wx-geeks/video/the-incredible-shrinking-cold-pool>-----
Pete Pokrandt - Systems Programmer
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086 - poker@xxxxxxxxxxxx
________________________________
From: Anne Myckow - NOAA Federal <anne.myckow@xxxxxxxx>
Sent: Friday, August 20, 2021 12:07 PM
To: Pete Pokrandt <poker@xxxxxxxxxxxx>
Cc: dustin.sheffler@xxxxxxxx <dustin.sheffler@xxxxxxxx>;
ncep.list.pmb-dataflow@xxxxxxxx <ncep.list.pmb-dataflow@xxxxxxxx>;
conduit@xxxxxxxxxxxxxxxx <conduit@xxxxxxxxxxxxxxxx>;
support-conduit@xxxxxxxxxxxxxxxx <support-conduit@xxxxxxxxxxxxxxxx>
Subject: Re: High CONDUIT latencies from vm-lnx-conduit2.ncep.noaa.gov
Hi Pete,
We have a lot of systems and applications running out of College Park right
now, which I think is part of it. But I will have someone take a look at
conduit2 today, see if maybe we need to try and move your connections to
conduit1 instead.
Thanks,
Anne
On Fri, Aug 20, 2021 at 12:54 PM Pete Pokrandt
<poker@xxxxxxxxxxxx<mailto:poker@xxxxxxxxxxxx>> wrote:
Dear Anne, Dustin and all,
Did you see this? We are still experiencing high latencies of 800-1000 seconds
on our CONDUIT feeds during the times when the GFS comes through that appear to
be coming from the host
vm-lnx-conduit2.ncep.noaa.gov<http://vm-lnx-conduit2.ncep.noaa.gov>
Here are the most recent lags. Any ideas?
Thanks,
Pete
https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd-agg.aos.wisc.edu
[cid:17b64872bebcb971f163]
https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+conduit.unidata.ucar.edu
[cid:17b64872bebcb971f164]
<http://www.weather.com/tv/shows/wx-geeks/video/the-incredible-shrinking-cold-pool>-----
Pete Pokrandt - Systems Programmer
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086 - poker@xxxxxxxxxxxx<mailto:poker@xxxxxxxxxxxx>
________________________________
From: Pete Pokrandt
Sent: Wednesday, August 18, 2021 3:02 PM
To: anne.myckow@xxxxxxxx<mailto:anne.myckow@xxxxxxxx>
<anne.myckow@xxxxxxxx<mailto:anne.myckow@xxxxxxxx>>;
dustin.sheffler@xxxxxxxx<mailto:dustin.sheffler@xxxxxxxx>
<dustin.sheffler@xxxxxxxx<mailto:dustin.sheffler@xxxxxxxx>>;
ncep.list.pmb-dataflow@xxxxxxxx<mailto:ncep.list.pmb-dataflow@xxxxxxxx>
<ncep.list.pmb-dataflow@xxxxxxxx<mailto:ncep.list.pmb-dataflow@xxxxxxxx>>
Cc: conduit@xxxxxxxxxxxxxxxx<mailto:conduit@xxxxxxxxxxxxxxxx>
<conduit@xxxxxxxxxxxxxxxx<mailto:conduit@xxxxxxxxxxxxxxxx>>;
support-conduit@xxxxxxxxxxxxxxxx<mailto:support-conduit@xxxxxxxxxxxxxxxx>
<support-conduit@xxxxxxxxxxxxxxxx<mailto:support-conduit@xxxxxxxxxxxxxxxx>>
Subject: High CONDUIT latencies from
vm-lnx-conduit2.ncep.noaa.gov<http://vm-lnx-conduit2.ncep.noaa.gov>
Dear Anne, Dustin and all,
Recently we have noticed fairly high latencies on the CONDUIT ldm data feed
originating from the machine
vm-lnx-conduit2.ncep.noaa.gov<http://vm-lnx-conduit2.ncep.noaa.gov>. The feed
originating from
vm-lnx-conduit1.ncep.noaa.gov<http://vm-lnx-conduit1.ncep.noaa.gov> does not
have the high latencies. Unidata and other top level feeds are seeing similar
high latencies from
vm-lnx-conduit2.ncep.noaa.gov<http://vm-lnx-conduit2.ncep.noaa.gov>.
Here are some graphs showing the latencies that I'm seeing:
>From
>https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd-agg.aos.wisc.edu
> - latencies for CONDUIT data arriving at our UW-Madison AOS ingest machine
[cid:17b64872bebcb971f161]
>From
>https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/siteindex?conduit.unidata.ucar.edu
> (latencies at Unidata)
[cid:17b64872bebcb971f162]
At least here at UW-Madison, these latencies are causing us to lose some data
during the large GFS/GEFS periods.
Any idea what might be causing this?
Pete
<http://www.weather.com/tv/shows/wx-geeks/video/the-incredible-shrinking-cold-pool>-----
Pete Pokrandt - Systems Programmer
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086 - poker@xxxxxxxxxxxx<mailto:poker@xxxxxxxxxxxx>
--
Anne Myckow
Dataflow Team Lead
NWS/NCEP/NCO
