Greetings fellow LDMers!
As you may recall, a certain Service Change Notice garnered some attention
a short time ago when some changes to the NOAAPort/SBN were announced.
Stonie and I have been investigating the potential impacts & what this will
mean for our community and we would like to share what we know.
*TL;DR:* We think we’ll be okay. We still have some questions ourselves,
and we won’t have all the answers until they flip the switch, but we
believe any impacts should be fairly limited for the most part. The best
thing users can do is be ready to take any necessary action should it be
needed that Monday morning.
*What’s all this then?*
SCN 24-54 (linked below) announces that as part of efforts to improve
dataflow, the NWS will “migrate all grib, grib2, and bufr formatted data
from the NMC channel to the NMC2 channel.” If you’re not familiar with
NMC/NMC2 lingo, these are data channels for the SBN that while similar to
LDM feedtypes do not exactly line up 1:1 with them. So what they’re doing
is moving some data from one of these channels to another.
https://www.weather.gov/media/notification/pdf_2023_24/scn24-54_channel_realignment_to_sbn_aaa.pdf
When LDM running NoaaportIngester receives SBN data, it does a series of
checks to determine which LDM feedtype the product should be filed into.
Again, it’s not a 1:1 relationship, and in places within the LDM code the
logic gets a little fuzzy. So for those of us running LDMs, whether it’s
at a NOAAPort receive site or requesting NOAAPort data relayed from another
site, we could see this data get filed differently.
*Dang it, Mike, I’m a Professor not an engineer! What does this mean for
me?*
- The data itself is NOT changing. We believe the worst-case scenario
here is we have to get used to new locations for data that may have moved
around.
- For most products we anticipate no change to how they are received by
the LDM. We believe the most likely adverse impact would be products
formerly seen on HDS to now appear on NGRID, however we will not know the
scale or selection of products until it happens.
- If you have REQUEST or PQACT entries that rely on feedtype-specific
expressions (e.g. HDS instead of ANY) then there is a chance products that
have been working could be missed, as they may now appear on a different
feed.
*What are the recommendations for LDM users?*
It’s going to boil down to what YOUR configurations look like that will
determine your best courses of action. In general though, here is our
advice at this time:
1. Take stock of your REQUEST and PQACT entries, especially those
involving the HDS and NGRID feedtypes.
2. Envision what your configurations would do if, for example, all HDS
data moved into NGRID. Would EXEC actions fire twice? Would files be
saved into improper directories?
3. *BE AWARE! Monday July 15th at or about 12Z the switch will be
flipped.* In the event something goes wrong (for them) they may
revert. If you have concerns you may want to be around to monitor the
rollout in case you need to take some action yourself.
*What about the SCN’s recommendation to change everything over to just use
ANY?*
Yeah, I was afraid you might bring that up. In general this actually isn’t
the worst advice. It may seem counter-intuitive, in fact it goes against
some guidance we’ve given users in the past. But in the spirit of ensuring
you get everything you need this would be the safest bet.
However, there may be side-effects of doing this. Whether or not there
are, or their potential impact, once again comes down to individual site
configurations. Here are some potential concerns you may consider:
- I want to believe that if everything went over to use ANY instead of
specific feedtypes that there would not be any overlap in Product IDs, but
I’m not sure we can guarantee that. While I have begun efforts to catalog
data on the SBN and IDD these tools are still some distance away from being
able to answer those questions with confidence.
- In other words, say you save every data product into the same
directory using the Product ID as its name (for the love of everything
don’t do this! This is a thought experiment!!). If we ignore feedtypes
entirely, would any data files be overwritten? We think the odds of this
are pretty small.
- Some sites have seen that breaking ANY requests into multiple requests
by feedtype results in significant performance improvements, in some cases
such a configuration is required for the installation to work properly. If
you know your site has such a concern, bear this in mind.
- Because we believe the potential impacts will be limited to the HDS
and NGRID feedtypes, it is likely a safer option to replace only those
REQUESTs with ANY instead of everything.
If you still have questions, feel free to reach out via our support
channels. See below for more information.
*A word about support:*
As always, if you have any questions comments or concerns that need to be
addressed in a timely manner, please do not hesitate to reach out to
support-idd@xxxxxxxxxxxxxxxx, support-ldm@xxxxxxxxxxxxxxxx,
support-noaaport@xxxxxxxxxxxxxxxx, or any other of our support inboxes
listed here: https://www.unidata.ucar.edu/support/index.html#request
Admittedly
it can be a little tough to choose at times, don’t worry about it too much
as they will all still ping the right people. But requesting support
through these inboxes really helps prevent things from falling through the
cracks (plus the tracking makes us look good for NSF 😉). That being said
here’s a guide:
- Support-idd for when you’re not getting data that you’re requesting
from us
- Support-ldm when the LDM software just done broke
- Support-noaaport if you are a NOAAPort receive site and are having
issues with that part of the pipeline
I will add that NSF Unidata staff are pretty loaded up these days, and it
may take a little longer for us to respond to support inquiries. We always
strive to address critical issues as soon as possible but it may not always
be right away. This is especially true during holidays and weekends; we
are not a 24/7 operation and do not always have the ability to address
issues until the next business day.
At the same time, we have noticed that the occasional support ticket (new
or a user response) doesn’t always alert us the way it should. We are
prioritizing getting that addressed but please never feel like we are
intentionally ignoring you. If at any time you feel your support inquiry
has not been received, please send an email to support@xxxxxxxxxxxxxxxx as
that inbox gets scrutinized a bit differently.
We ask that you try to write staff directly only as a last resort unless
otherwise advised. It’s not that we don’t want to hear from you, but
sometimes tapping on the glass startles the engineers. In all seriousness
sometimes people go on PTO or other such things, and this can lead to
delays in getting your issues addressed. Thank you for attending my NSF
Unidata Support PSA. 😊
*Monday July 15th at 12Z.* I’ll be all coffee’d up and ready to go, and if
stuff approaches ceiling fan levels I’ll do my best to keep you posted.
Best,
-Mike
*Mike Zuranski*
Data Engineer II
NSF Unidata Program Center
University Corporation for Atmospheric Research
