Thanks, Ray! Isn't that just lovely...
Gilbert
> On Mar 14, 2017, at 3:55 PM, "admin@xxxxxxxx" <admin@xxxxxxxx> wrote:
>
> This is common in enterprise switches for that filtering to be ON by default,
> after some malware appeared that used multicast packets to do denial of
> service.
>
> Ray Weber
>
> On Tuesday, March 14, 2017 4:50pm, "Gilbert Sebenste" <gilbert@xxxxxxx> said:
>
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web. Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/ Hey Gerry,
>>
>> It’s an Avaya 5510 (not 5520, sorry---the one has half of the number of
>> ports, but is otherwise the same as the 20).
>> By default, it rejects the flood of packets on those encapsulated IP’s. And
>> I now realize, COD isn’t the only one having this issue…
>>
>> Gilbert
>>
>> Gilbert Sebenste
>> Staff Meteorologist
>>
>> Environmental Health and Safety
>> Labs for Wellness 154 | DeKalb, Illinois 60115
>> 815-753-5492
>> gilbert@xxxxxxx<mailto:gilbert@xxxxxxx>
>> http://weather.admin.niu.edu<http://weather.admin.niu.edu/>
>>
>> Everyone. Home. Safely.
>>
>> [NIU]
>>
>>
>> From: Gerry Creager - NOAA Affiliate [mailto:gerry.creager@xxxxxxxx]
>> Sent: Tuesday, March 14, 2017 3:38 PM
>> To: Gilbert Sebenste <gilbert@xxxxxxx>
>> Cc: ldm-users@xxxxxxxxxxxxxxxx; NOAAPORT <noaaport@xxxxxxxxxxxxxxxx>
>> Subject: Re: [ldm-users] GOES-16 reception issues when going through an
>> enterprise
>> level switch
>>
>> Who'd have thought the switch wouldn't pass multicast? Is it possible the
>> campus
>> network geeks told it not to? Using multicast for disseminating data like
>> that
>> "just makes sense".
>> Glad you found THAT problem. What brand of switch, anyway? Cisco?
>> gerry
>>
>> On Tue, Mar 14, 2017 at 3:30 PM, Gilbert Sebenste
>> <gilbert@xxxxxxx<mailto:gilbert@xxxxxxx>> wrote:
>> Hello everyone,
>>
>> The College of DuPage has a NOAAport receiver with a Novra S300N, and two
>> Ubuntu
>> boxes as ingestors, so that if one fails, the other will still provide data.
>> To
>> accomplish this, they use an enterprise level network switch, a Nortel/Avaya
>> 5520,
>> that costs a lot of money (and not the $50 gigabit switches you can get from
>> Newegg, or wherever), to split the data feed so that both servers can get and
>> ultimately relay the data.
>>
>> When trying to receive the test GOES-16 imagery months ago, it wasn’t coming
>> across their NOAAport feed. I thought it was a glitch with their Novra box,
>> or the
>> network switch. But now that GOES-16 data is being sent, neither NOAAPort
>> ingest
>> server were getting the GOES-16 data on the NOTHER feed. What could be wrong?
>>
>> We restarted the LDM on both servers, deleting and remaking the queues. That
>> didn’t do anything.
>> We then double checked the ldmd.conf files, which had a few minor issues,
>> and also
>> checked the logs,
>> of course. Still, no GOES-16 data was showing up.
>> We then did a packet sniff from one of the servers to the switch. No GOES-16
>> test
>> data was being sent to the boxes from the switch!
>>
>> We then made sure the Ubuntu kernel parameters were set correctly, per the
>> LDM
>> instructions. One box didn’t have it correct; the other was correct. We
>> rebooted both boxes after doing checks. That didn’t get us GOES-16 data,
>> either.
>>
>> So then, today, I convinced a College of DuPage meteorology program employee
>> to go
>> out to the bitter cold dish farm and climate-controlled “dog house”
>> where the Novra and the servers are. As he power-cycled the Novra, I was
>> doing an
>> ldmadmin watch on the NOTHER feed on the ingest servers. When the Novra came
>> back
>> up after the power cycke…no GOES-16 data was coming across to the servers!
>>
>> So then we looked to the network switch. The employee grabbed a jumper
>> cable, and
>> ran an Ethernet cable directly from the Novra to one of the two ingest
>> servers.
>> Then, finally, GOES-16 data started pouring in on that server!
>>
>> But why? How can a $1,000+ enterprise level switch have a problem like this?
>>
>> Well, as it turns out, it wasn’t the fault of the switch at all. See:
>> 224.0.1.[1-10] have reserved internet protocols on them:
>> http://www.networksorcery.com/enp/protocol/ip/multicast.htm
>>
>> For those of you who don’t like to understandably click on links from
>> strangers (and geeky meteorologists):
>>
>> INTERNETWORK
>> CONTROL BLOCK
>>
>> 224.0.1.0
>>
>> VMTP<http://www.networksorcery.com/enp/protocol/vmtp.htm> Managers.
>>
>> RFC 1045
>>
>> 224.0.1.1
>>
>> NTP<http://www.networksorcery.com/enp/protocol/ntp.htm>, Network Time
>> Protocol.
>>
>> RFC 1119<http://www.networksorcery.com/enp/rfc/rfc1119.pdf>
>>
>> 224.0.1.2
>>
>> SGI-Dogfight.
>>
>>
>>
>> 224.0.1.3
>>
>> Rwhod.
>>
>>
>>
>> 224.0.1.4
>>
>> VNP.
>>
>>
>>
>> 224.0.1.5
>>
>> Artificial Horizons - Aviator.
>>
>>
>>
>> 224.0.1.6
>>
>> NSS, Name Service Server.
>>
>>
>>
>> 224.0.1.7
>>
>> AUDIONEWS - Audio News Multicast.
>>
>>
>>
>> 224.0.1.8
>>
>> SUN NIS+ Information Service.
>>
>>
>>
>> 224.0.1.9
>>
>> MTP<http://www.networksorcery.com/enp/protocol/mtp.htm>, Multicast Transport
>> Protocol.
>>
>> RFC 1301<http://www.networksorcery.com/enp/rfc/rfc1301.txt>
>>
>> 224.0.1.10
>>
>> IETF-1-LOW-AUDIO.
>>
>>
>> Uh huh. When the NWS picked those IP addresses to encapsulate the data, they
>> picked IP numbers that are used and reserved for enterprise-level switches.
>>
>> As for the College of DuPage? I just got this email from head network
>> engineer,
>> Dave Bukowski, who came up with this solution: you need to enable unknown
>> multicast flooding on those IP’s. Here’s what Dave did, and his email
>> with the solution:
>> -----------
>>
>> FOUND IT!!!!
>>
>>
>> This was in the config
>>
>> show running-config
>> .....SNIP.....
>> !
>> ! *** IGMP ***
>> !
>> vlan igmp unknown-mcast-allow-flood 224.0.1.1
>> vlan igmp unknown-mcast-allow-flood 224.0.1.2
>> vlan igmp unknown-mcast-allow-flood 224.0.1.3
>> vlan igmp unknown-mcast-allow-flood 224.0.1.4
>> vlan igmp unknown-mcast-allow-flood 224.0.1.5
>> vlan igmp unknown-mcast-no-flood enable
>> .....ENDSNIP.....
>>
>>
>> Added
>> vlan igmp unknown-mcast-allow-flood 224.0.1.6
>> vlan igmp unknown-mcast-allow-flood 224.0.1.7
>> vlan igmp unknown-mcast-allow-flood 224.0.1.8
>> vlan igmp unknown-mcast-allow-flood 224.0.1.9
>> vlan igmp unknown-mcast-allow-flood 224.0.1.10
>> vlan igmp unknown-mcast-allow-flood 224.0.1.201
>>
>>
>>
>> So now:
>>
>> SDBDF01#show vlan igmp unknown-mcast-allow-flood
>> Allowed Multicast Addresses
>> -----------------------------------------------------------------------
>> 224.0.1.1 224.0.1.2 224.0.1.3 224.0.1.4
>> 224.0.1.5 224.0.1.6 224.0.1.7 224.0.1.8
>> 224.0.1.9 224.0.1.10 224.0.1.201
>> Total Multicast MAC Addresses: 0
>> Total Multicast IP Addresses: 11
>>
>>
>> Now we have a flood of data coming in on both noaaport1 and noaaport2. It was
>> multicast and we were blocking the flood, but had to specifically allow the
>> flooding by multicast address.
>>
>> -Dave
>>
>>
>> Thanks, Dave. We spent at least 10 hours trying to figure this out. On a $50
>> switch, you might not have a problem, but if you do decide to have multiple
>> ingesters by going through a network switch…if you don’t get all of
>> the data, your reception may be fine…it’s the switch that’s
>> been programmed to stop the flow of data.
>>
>> Thanks to Tom Yoksas as he helped us try to figure this mess out, to Mike
>> Zuranski
>> for going out in the bitter cold and spending hours on the phone as we tried
>> to
>> figure this out, and for Dave Bukowski who ran the ball for the touchdown
>> after we
>> discovered it was the switch blocking the data.
>>
>> Gilbert
>> Gilbert Sebenste
>> Staff Meteorologist
>>
>> Environmental Health and Safety
>> Labs for Wellness 154 | DeKalb, Illinois 60115
>> 815-753-5492<tel:(815)%20753-5492>
>> gilbert@xxxxxxx<mailto:gilbert@xxxxxxx>
>> http://weather.admin.niu.edu<http://weather.admin.niu.edu/>
>> Everyone. Home. Safely.
>>
>> [NIU]
>>
>>
>>
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web. Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>>
>>
>> --
>> Gerry Creager
>> NSSL/CIMMS
>> 405.325.6371
>> ++++++++++++++++++++++
>> “Big whorls have little whorls,
>> That feed on their velocity;
>> And little whorls have lesser whorls,
>> And so on to viscosity.”
>> Lewis Fry Richardson (1881-1953)
>>
>
>
