Good morning all,
The SCN has been updated to push back the implementation date from July
15th to August 5th, giving us another three weeks before these changes
are made:
https://www.weather.gov/media/notification/pdf_2023_24/scn24-54_channel_realignment_to_sbn_aab.pdf
The guidance within hasn't really changed, though I did find it interesting
this will be the first of three changes while the other two are still a
mystery... to me anyways.
We continue to stand by our messaging sent out yesterday, as only the
timing of this has been updated and not anything else. We will continue to
pay close attention to this SCN and will relay any updates and guidance as
best as we can.
Thanks,
-Mike
*Mike Zuranski*
Data Engineer II
NSF Unidata Program Center
University Corporation for Atmospheric Research
On Mon, Jul 8, 2024 at 10:20 AM Mike Zuranski <mzuranski@xxxxxxxx> wrote:
> Greetings fellow LDMers!
>
> As you may recall, a certain Service Change Notice garnered some attention
> a short time ago when some changes to the NOAAPort/SBN were announced.
> Stonie and I have been investigating the potential impacts & what this will
> mean for our community and we would like to share what we know.
>
> *TL;DR:* We think we’ll be okay. We still have some questions
> ourselves, and we won’t have all the answers until they flip the switch,
> but we believe any impacts should be fairly limited for the most part. The
> best thing users can do is be ready to take any necessary action should it
> be needed that Monday morning.
>
>
> *What’s all this then?*
>
> SCN 24-54 (linked below) announces that as part of efforts to improve
> dataflow, the NWS will “migrate all grib, grib2, and bufr formatted data
> from the NMC channel to the NMC2 channel.” If you’re not familiar with
> NMC/NMC2 lingo, these are data channels for the SBN that while similar to
> LDM feedtypes do not exactly line up 1:1 with them. So what they’re doing
> is moving some data from one of these channels to another.
> https://www.weather.gov/media/notification/pdf_2023_24/scn24-54_channel_realignment_to_sbn_aaa.pdf
>
> When LDM running NoaaportIngester receives SBN data, it does a series of
> checks to determine which LDM feedtype the product should be filed into.
> Again, it’s not a 1:1 relationship, and in places within the LDM code the
> logic gets a little fuzzy. So for those of us running LDMs, whether it’s
> at a NOAAPort receive site or requesting NOAAPort data relayed from another
> site, we could see this data get filed differently.
>
>
> *Dang it, Mike, I’m a Professor not an engineer! What does this mean for
> me?*
>
> - The data itself is NOT changing. We believe the worst-case scenario
> here is we have to get used to new locations for data that may have moved
> around.
> - For most products we anticipate no change to how they are received
> by the LDM. We believe the most likely adverse impact would be products
> formerly seen on HDS to now appear on NGRID, however we will not know the
> scale or selection of products until it happens.
> - If you have REQUEST or PQACT entries that rely on feedtype-specific
> expressions (e.g. HDS instead of ANY) then there is a chance products that
> have been working could be missed, as they may now appear on a different
> feed.
>
>
> *What are the recommendations for LDM users?*
>
> It’s going to boil down to what YOUR configurations look like that will
> determine your best courses of action. In general though, here is our
> advice at this time:
>
> 1. Take stock of your REQUEST and PQACT entries, especially those
> involving the HDS and NGRID feedtypes.
> 2. Envision what your configurations would do if, for example, all HDS
> data moved into NGRID. Would EXEC actions fire twice? Would files be
> saved into improper directories?
> 3. *BE AWARE! Monday July 15th at or about 12Z the switch will be
> flipped.* In the event something goes wrong (for them) they may
> revert. If you have concerns you may want to be around to monitor the
> rollout in case you need to take some action yourself.
>
>
> *What about the SCN’s recommendation to change everything over to just use
> ANY?*
>
> Yeah, I was afraid you might bring that up. In general this actually
> isn’t the worst advice. It may seem counter-intuitive, in fact it goes
> against some guidance we’ve given users in the past. But in the spirit of
> ensuring you get everything you need this would be the safest bet.
>
> However, there may be side-effects of doing this. Whether or not there
> are, or their potential impact, once again comes down to individual site
> configurations. Here are some potential concerns you may consider:
>
> - I want to believe that if everything went over to use ANY instead of
> specific feedtypes that there would not be any overlap in Product IDs, but
> I’m not sure we can guarantee that. While I have begun efforts to catalog
> data on the SBN and IDD these tools are still some distance away from being
> able to answer those questions with confidence.
> - In other words, say you save every data product into the same
> directory using the Product ID as its name (for the love of everything
> don’t do this! This is a thought experiment!!). If we ignore feedtypes
> entirely, would any data files be overwritten? We think the odds of this
> are pretty small.
> - Some sites have seen that breaking ANY requests into multiple
> requests by feedtype results in significant performance improvements, in
> some cases such a configuration is required for the installation to work
> properly. If you know your site has such a concern, bear this in mind .
> - Because we believe the potential impacts will be limited to the HDS
> and NGRID feedtypes, it is likely a safer option to replace only those
> REQUESTs with ANY instead of everything.
>
>
> If you still have questions, feel free to reach out via our support
> channels. See below for more information.
>
>
> *A word about support:*
>
> As always, if you have any questions comments or concerns that need to be
> addressed in a timely manner, please do not hesitate to reach out to
> support-idd@xxxxxxxxxxxxxxxx, support-ldm@xxxxxxxxxxxxxxxx,
> support-noaaport@xxxxxxxxxxxxxxxx, or any other of our support inboxes
> listed here: https://www.unidata.ucar.edu/support/index.html#request
> Admittedly
> it can be a little tough to choose at times, don’t worry about it too much
> as they will all still ping the right people. But requesting support
> through these inboxes really helps prevent things from falling through the
> cracks (plus the tracking makes us look good for NSF 😉). That being said
> here’s a guide:
>
> - Support-idd for when you’re not getting data that you’re requesting
> from us
> - Support-ldm when the LDM software just done broke
> - Support-noaaport if you are a NOAAPort receive site and are having
> issues with that part of the pipeline
>
>
> I will add that NSF Unidata staff are pretty loaded up these days, and it
> may take a little longer for us to respond to support inquiries. We always
> strive to address critical issues as soon as possible but it may not always
> be right away. This is especially true during holidays and weekends; we
> are not a 24/7 operation and do not always have the ability to address
> issues until the next business day.
>
> At the same time, we have noticed that the occasional support ticket (new
> or a user response) doesn’t always alert us the way it should. We are
> prioritizing getting that addressed but please never feel like we are
> intentionally ignoring you. If at any time you feel your support inquiry
> has not been received, please send an email to support@xxxxxxxxxxxxxxxx as
> that inbox gets scrutinized a bit differently.
>
> We ask that you try to write staff directly only as a last resort unless
> otherwise advised. It’s not that we don’t want to hear from you, but
> sometimes tapping on the glass startles the engineers. In all seriousness
> sometimes people go on PTO or other such things, and this can lead to
> delays in getting your issues addressed. Thank you for attending my NSF
> Unidata Support PSA. 😊
>
>
> *Monday July 15th at 12Z.* I’ll be all coffee’d up and ready to go, and
> if stuff approaches ceiling fan levels I’ll do my best to keep you posted.
>
> Best,
> -Mike
>
>
> *Mike Zuranski*
> Data Engineer II
> NSF Unidata Program Center
> University Corporation for Atmospheric Research
>
