Updates:
*SENIOR DUTY METEOROLOGIST NWS ADMINISTRATIVE MESSAGENWS NCEP CENTRAL
OPERATIONS COLLEGE PARK MD0311Z TUE DEC 14 2021...MISSING/SHORT PRODUCTS
SENT OUT OVER THE SBN...NCF has been investigating missing/short products
being sent outover the SBN today. They have just switched over the SBN
uplinkto the HMGS which should resolve this issue. *
Since then the issue appears to have been resolved. Data flow looks to
have stabilized and our imagery looks normal.
After the switch I took another look and the source IP on NOAAPort is back
to 10.0.9.51. Still no idea if it's related, but there's a data point.
-Mike
======================
Mike Zuranski
Meteorology Support Analyst
College of DuPage - Nexlab
Weather.cod.edu <http://weather.cod.edu/>
======================
On Mon, Dec 13, 2021 at 5:50 PM Mike Zuranski <zuranski.wx@xxxxxxxxx> wrote:
> Thanks Gilbert.
>
> Someone from OU just reached out to me with an interesting observation...
>
> *We had an outage on our NOAAport receive system this morning, which
> caused our systems to fail over to our partner organization's feeds. On
> investigation, we found that our receiver, ingest software, and LDM
> software were all working fine. We saw packets coming into the Linux-based
> ingest server, but nothing was being ingested. Then, it was noticed that
> the source IP of the NOAAport multicast traffic had changed, for at least
> some of the packets. Whereas in the past, it had been 10.0.9.51 , we are
> now seeing packets from 10.0.9.52 . Our ingest server was not accepting
> those because it had no special routing rule for that IP. We added such a
> rule (and did *not* delete the existing one for the .51 IP) and our
> NOAAport ingest began working as before.*
>
>
> I can confirm the source IP for me is 10.0.9.52 as well, but unlike him
> that IP range was already open on my system and that traffic was making its
> way in. I'm not sure if this is related or not, but I wanted to pass along
> the intel in case it is.
>
> -Mike
>
> ======================
> Mike Zuranski
> Meteorology Support Analyst
> College of DuPage - Nexlab
> Weather.cod.edu <http://weather.cod.edu/>
> ======================
>
>
> On Mon, Dec 13, 2021 at 5:07 PM Gilbert Sebenste <gilbert@xxxxxxxxxxxxxxxx>
> wrote:
>
>> I have put in a trouble ticket with NCO. I'll keep everyone posted.
>>
>> Gilbert
>>
>> On Dec 13, 2021, at 3:11 PM, Mike Zuranski <zuranski.wx@xxxxxxxxx> wrote:
>>
>>
>> Hi Jim,
>>
>> I was just about to write my own email about this. It seems to me that
>> not all GOES-16 tiles are making it to NOAAPort. I was about to ask if
>> anyone else is seeing this, and I guess you are.
>>
>> I noticed this began around 14Z today. GOES-16 CONUS & Full-Disk are
>> impacted, Mesoscale and GOES-17 seem okay. I haven't seen any
>> announcements about data flow issues. Using notifyme I saw that Unidata
>> was missing the same tiles I was.
>>
>> Does anyone else have any insight?
>>
>> -Mike
>>
>> ======================
>> Mike Zuranski
>> Meteorology Support Analyst
>> College of DuPage - Nexlab
>> Weather.cod.edu <http://weather.cod.edu/>
>> ======================
>>
>>
>> On Mon, Dec 13, 2021 at 3:06 PM Weber, Jim <Jim.Weber@xxxxxxx> wrote:
>>
>>> I have been using the GOES Restitch for a couple of months now and it
>>> has worked perfectly until today. Today I am getting partial images. Upon
>>> looking at the log files I am finding that it is “Timing Out” waiting for
>>> the tile to arrive. I am not seeing any large amounts of dropped packets.
>>> I am not sure what could be causing my sudden issue. Any insight as to
>>> what I might check?
>>>
>>>
>>>
>>> Thanks, Jim ( partial log below)
>>>
>>>
>>>
>>>
>>>
>>> [TIRE 132051] 2021-12-13 20:54:52,378 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C05_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 8 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:54:53,177 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C01_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 7 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:54:54,033 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C01_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 8 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:00,091 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C03_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 11 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:00,753 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C01_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 9 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:01,554 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C03_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 12 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:06,066 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C02_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 6 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:06,350 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C03_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 13 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:07,089 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C05_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 9 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:07,592 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C01_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 10 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:08,514 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C05_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 10 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:13,149 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C02_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 7 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:16,471 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C02_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 8 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:22,188 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C02_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 9 of 15 tiles received.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:34,664 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C02_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 10 of 15 tiles received.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:44,834 [process_items]: Finishing due to
>>> timeout.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:44,944 [store_done]:
>>> GOES16/Products/CloudAndMoistureImagery/CONUS/Channel07/20211213/OR_ABI-L2-CMIPC-M6C07_G16_s20213472046170_e20213472046170_c20213472046170.nc
>>> finished.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:44,947 [process_items]: Finishing due to
>>> timeout.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:45,051 [store_done]:
>>> GOES16/Products/CloudAndMoistureImagery/CONUS/Channel12/20211213/OR_ABI-L2-CMIPC-M6C12_G16_s20213472046170_e20213472046170_c20213472046170.nc
>>> finished.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:45,698 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C02_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 11 of 15 tiles received.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:47,772 [process_items]: Finishing due to
>>> timeout.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:47,875 [store_done]:
>>> GOES16/Products/CloudAndMoistureImagery/CONUS/Channel10/20211213/OR_ABI-L2-CMIPC-M6C10_G16_s20213472046170_e20213472046170_c20213472046170.nc
>>> finished.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:48,075 [process_items]: Finishing due to
>>> timeout.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:48,181 [store_done]:
>>> GOES16/Products/CloudAndMoistureImagery/CONUS/Channel11/20211213/OR_ABI-L2-CMIPC-M6C11_G16_s20213472046170_e20213472046170_c20213472046170.nc
>>> finished.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:49,455 [process_items]: Finishing due to
>>> timeout.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:49,559 [store_done]:
>>> GOES16/Products/CloudAndMoistureImagery/CONUS/Channel09/20211213/OR_ABI-L2-CMIPC-M6C09_G16_s20213472046170_e20213472046170_c20213472046170.nc
>>> finished.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:49,560 [process_items]: Finishing due to
>>> timeout.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:49,664 [store_done]:
>>> GOES16/Products/CloudAndMoistureImagery/CONUS/Channel08/20211213/OR_ABI-L2-CMIPC-M6C08_G16_s20213472046170_e20213472046170_c20213472046170.nc
>>> finished.
>>>
>>> [TIRE 132051] 2021-12-13 20:55:49,891 [process_items]:
>>> OR_ABI-L2-CMIPC-M6C02_G16_s20213472051170_e20213472051170_c20213472051170.nc:
>>> 12 of 15 tiles received.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:51,447 [process_items]: Finishing due to
>>> timeout.
>>>
>>> [TIRE 132046] 2021-12-13 20:55:51,556 [store_done]:
>>> GOES16/Products/CloudAndMoistureImagery/CONUS/Channel13/20211213/OR_ABI-L2-CMIPC-M6C13_G16_s20213472046170_e20213472046170_c20213472046170.nc
>>> finished.
>>>
>>>
>>>
>>>
>>> This message and its attachments may contain legally privileged or
>>> confidential information. It is intended solely for the named addressee(s).
>>> If you are not an addressee indicated in this message (or responsible for
>>> delivery of the message to an addressee), you may not copy or deliver this
>>> message or its attachments to anyone. Rather, you should permanently delete
>>> this message and its attachments and kindly notify the sender by reply
>>> e-mail. Any content of this message and its attachments that does not
>>> relate to the official business of Fox Corporation, or its subsidiaries
>>> must be taken not to have been sent or endorsed by any of them. No
>>> representation is made that this email or its attachments are without
>>> defect.
>>> _______________________________________________
>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>> recorded in the Unidata inquiry tracking system and made publicly
>>> available through the web. Users who post to any of the lists we
>>> maintain are reminded to remove any personal information that they
>>> do not want to be made public.
>>>
>>>
>>> python-users mailing list
>>> python-users@xxxxxxxxxxxxxxxx
>>> For list information, to unsubscribe, or change your membership options,
>>> visit: https://www.unidata.ucar.edu/mailing_lists/
>>>
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web. Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> https://www.unidata.ucar.edu/mailing_lists/
>>
>>
