[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [conduit] Large CONDUIT latencies to UW-Madison idd.aos.wisc.edu starting the last day or two.



So from the looks of your screen grab that is 7 minutes to download the 0.25 GFS from our FTP server. Do you have ideas of how slow that is compared to normal from Colorado? I will be passing this information on to our IT folks.


Carissa Klemmer
NCEP Central Operations
Dataflow Team Lead

Hey Carissa J

 

I left the fetch script run on our colo box this morning so we could get a good idea of download times. Here is a screen cap of that directory structure… Essentially when I fetch model files, I drop them on a ramdisk and encode them on the fly, and then leave them on the ramdisk to ensure faster processing. Here is what the 06z run looks like so far (obviously non-operational J )

 

http://drmalachi.org/files/ncep/he-gfsdl.2016.02.22.png

 

So we can see times vary from 6 minutes to it looks like 26 minutes to download one GFS run from a 100gbit connection directly on the Hurricane Electric backbone 8 hops away from ncep J  Here is the traceroute / MTR from that box earlier this morning:

 

http://drmalachi.org/files/ncep/he-ncep.2016.02.22.jpg

 

What leads me to believe that packetshaping is the likely culprit, is that I can jump on my amazon ec2 box at the same time, and download the same file in about 2 seconds from the same source J

 

Here is a sample of the routing from the EC2 spinup to the ncep server.. recall the first 20 or so hops are just internal to amazon

 

http://drmalachi.org/files/ncep/ec2-ncep.png

 

The difference between the Hurricane Electric / colo routing to the amazon routing appears to be the handoff / routing from Internet2 to gigapop to 140.90.111.36… so whatever is “triggering” the different routes is what is initiating the delay J   Or it could just be the dreaded gopher! J

 

Happy Monday!

 

Cheers,

 

--patrick

 

-------------------------------------------------------

Patrick L. Francis

Vice President of Research & Development

 

Aeris Weather

 

http://aerisweather.com/

http://modelweather.com/

 

http://facebook.com/wxprofessor/

 

 

 

 

_______________________________________________
conduit mailing list
address@hidden
For list information or to unsubscribe, visit: 
http://www.unidata.ucar.edu/mailing_lists/