[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[CONDUIT #PNI-665795]: 20150331: high CONDUIT latencies

Hi Carissa,

I am assuming that you are aware of traffic on the address@hidden
email list today that is centered around unexpectedly high CONDUIT latencies
in feeds originating from ncepldm4.woc.noaa.gov and conduit.ncep.noaa.gov.
I wanted to touch base with you and the NCEP Data Flow team to point out
something that did not come to light in the discussions on the list:

It is interesting to compare latencies at the University of Wisconsin/AOS
(idd.aos.wisc.edu) to those being experienced by us (Unidata) in Boulder,
CO and and Penn State in State College, PA:

daffy.unidata.ucar.edu (the CONDUIT accumulator here in the UPC):
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+daffy.unidata.ucar.edu

idd.meteo.psu.edu
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd.meteo.psu.edu

idd.aos.wisc.edu
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd.aos.wisc.edu

As you can seefrom these plots, the UW/AOS latencies are much worse than either 
PSU's
or ours. I interpret this to mean that some part of the problem is closer to UW 
than
it is to NCEP (but I could be wrong).

What is _really_ interesting (and goes against the conjecture in the previous
sentence) is the latencies we are seeing in the test CONDUIT feed that contains
all of the standard CONDUIT feed plus the 0.25 degree GFS data:

lead.unidata.ucar.edu
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+lead.unidata.ucar.edu

Even though the data volumes in the test CONDUIT feed are up to twice what they
are in the operational feed, the latencies are _much_ lower!  This seems weird
since I believe that the NCEP machine (IP address 140.90.101.93) that is feeding
lead.unidata.ucar.edu is located in the same datacenter (and perhaps even on the
same physical machine since it looks like a VM) as the cluster that is relaying
the operational feed. Why latencies from this single machine would be lower is a
mystery especially since we are using a single feed REQUEST for the higher 
volume,
test CONDUIT feed on lead.unidata.ucar.edu and split feed REQUESTs (5 mutually
exclusive REQUESTs) on our CONDUIT accumulator, daffy.unidata.ucar.edu.

I don't know if the markedly better deliver of the test CONDUIT datastream 
will help to figure out what my be going on in the operational feed, but I
figured I would pass along the information nonetheless.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************