Stonie, my man! Thank you for conjuring that up so fast!
Best,
-Z
On Wed, Jan 29, 2025 at 11:54 AM Stonie Cooper <cooper@xxxxxxxx> wrote:
> Sorry, included an echo you don't need. Should be:
>
> --- cut here ---
> #!/bin/bash
> #assessing the next hour based upon 10 minute look forward as described by
> NOAA
> datahome="${1}"
> datafile="${2}"
> minutes="${3}"
>
> mkdir -p ${datahome}
> dateonly=`echo "${datafile}" | grep -E -o "[0-9]*"`
>
> if [ "${#dateonly}" -eq "8" ]; then
> outputdate=`date -d "20${datafile:0:6} ${datafile:6:2}:${minutes}:00 Z
> +10minutes" -u "+%y%m%d%H"`
> elif [ "${#dateonly}" -eq "10" ]; then
> outputdate=`date -d "${datafile:0:8} ${datafile:8:2}:${minutes}:00 Z
> +10minutes" -u "+%Y%m%d%H"`
> else
> exit
> fi
>
> outputfile=`echo "${datafile}" | sed -e "s|${dateonly}|${outputdate}|g"`
>
> cat >> ${datahome}/${outputfile}
> --- cut here ---
> -
> Stonie Cooper, PhD
> Software Engineer III
> NSF Unidata
> cooper@xxxxxxxx
>
>
> On Wed, Jan 29, 2025 at 11:50 AM Stonie Cooper <cooper@xxxxxxxx> wrote:
>
>> Using Pete's example as the template:
>>
>> DDS|IDS ^S[AP].* .... ([0-3][0-9])([0-2][0-9])
>> FILE /usr/local/ldm/data/weather/(\1:yy)(\1:mm)\1\2.METAR
>>
>> Which works for everything but the new messages, for the new 99 headed
>> data, *an* option:
>>
>> DDS|IDS ^S[AP]US99 .... ([0-3][0-9])([0-2][0-9])([0-5][0-9])
>> PIPE /usr/local/ldm/util/saus99_binning.sh
>> /usr/local/ldm/data/weather/ (\1:yy)(\1:mm)\1\2.METAR \3
>>
>> Where /usr/local/ldm/util/saus99_binning.sh *could* be something like:
>>
>> --- cut here ---
>> #!/bin/bash
>> #assessing the next hour based upon 10 minute look forward as described
>> by NOAA
>> datahome="${1}"
>> datafile="${2}"
>> minutes="${3}"
>>
>> mkdir -p ${datahome}
>> dateonly=`echo "${datafile}" | grep -E -o "[0-9]*"`
>> echo "${#dateonly}"
>>
>> if [ "${#dateonly}" -eq "8" ]; then
>> outputdate=`date -d "20${datafile:0:6} ${datafile:6:2}:${minutes}:00 Z
>> +10minutes" -u "+%y%m%d%H"`
>> else
>> outputdate=`date -d "${datafile:0:8} ${datafile:8:2}:${minutes}:00 Z
>> +10minutes" -u "+%Y%m%d%H"`
>> fi
>>
>> outputfile=`echo "${datafile}" | sed -e "s|${dateonly}|${outputdate}|g"`
>>
>> cat >> ${datahome}/${outputfile}
>> --- cut here ---
>>
>> If you do this, you will want to edit your original pattern to exclude
>> the S[AP]US99 . . . for everything else.
>>
>> Or you can pipe everything through the script.
>> -
>> Stonie Cooper, PhD
>> Software Engineer III
>> NSF Unidata
>> cooper@xxxxxxxx
>>
>>
>> On Wed, Jan 29, 2025 at 11:07 AM Mike Zuranski <zuranski.wx@xxxxxxxxx>
>> wrote:
>>
>>> Morning all,
>>>
>>> I received a response to my inquiry:
>>>
>>> First thank you for reaching out to us on this matter. *This was an
>>> unintentional change* but I can describe how it happened. The SAUS70 &
>>> SAUS80 METAR collectives that were previously disseminated over the
>>> SBN/NOAAPort prior to this change would get disseminated every 10 minutes,
>>> including the latest observations in that period. For example, many
>>> observations are time stamped at 55 after the hour, which are then included
>>> in the top of the hour SAUS70 METAR collective. However, SAUS99 METAR
>>> collectives are issued upwards of several times a minute containing the
>>> latest observation. We get a big benefit here in more timely observational
>>> data. In addition to the improved timeliness, SAUS70 & SAUS80 require
>>> manual intervention to add sites whereas sites automatically are added to
>>> SAUS99. Given these two benefits, *we hope the end-users applications
>>> are able to be updated to work with SAUS99*.
>>>
>>> Feel free to let us know if you have any additional questions or
>>> concerns!
>>>
>>> So there it is, this is the new norm and expected behavior moving
>>> forward.
>>>
>>> If this is negatively impactful to community members enough, my personal
>>> opinion is this might be addressed the easiest if Unidata could offer a
>>> decoder of sorts to adapt to this new norm. I can't say how widespread any
>>> impacts are, all I've heard is what's been said here. And of course I'm
>>> sure Unidata staff are hard at work as always and it's not my place to
>>> suggest how their energies should be spent. But that's where my head is
>>> heading.
>>>
>>> Best,
>>> -Mike
>>>
>>> On Tue, Jan 28, 2025 at 2:47 PM Sebenste, Gilbert <sebensteg@xxxxxxx>
>>> wrote:
>>>
>>>> This is what I see with the raw NADIN/WMSCR feed, which I’ve had access
>>>> to. These are indeed the RAW METARs sent without any processing from the
>>>> NWS, which slowed things down and had duplicate observations. The upsides
>>>> are that new METARs show up immediately, and assuming the feed to the NWS
>>>> is timely, you’ll get them 1-10 minutes faster than before.
>>>>
>>>>
>>>>
>>>> Gilbert Sebenste
>>>>
>>>> Meteorology Support Analyst
>>>>
>>>>
>>>>
>>>> *From:* ldm-users <ldm-users-bounces@xxxxxxxxxxxxxxxx> *On Behalf Of *Pete
>>>> Pokrandt
>>>> *Sent:* Monday, January 27, 2025 2:20 PM
>>>> *To:* Herzmann, Daryl E [AGRON] <akrherz@xxxxxxxxxxx>;
>>>> ldm-users@xxxxxxxxxxxxxxxx; Tyle, Kevin R <ktyle@xxxxxxxxxx>
>>>> *Subject:* [External] Re: [ldm-users] The S[AP]US99 update for METARs
>>>> just happened
>>>>
>>>>
>>>>
>>>> CAUTION: This email originated from outside of COD’s system. Do not
>>>> click links, open attachments, or respond with sensitive information unless
>>>> you recognize the sender and know the content is safe.
>>>>
>>>>
>>>>
>>>> All,
>>>>
>>>>
>>>>
>>>> I just noticed a curiosity about the new SAUS99 METAR format that I'm
>>>> not immediately sure how to deal with. I am getting the SAUS99 and SPUS99
>>>> reports, but the time on the WMO header is different, and that's messing up
>>>> my data filing in the LDM.
>>>>
>>>>
>>>>
>>>> To save data for the old Pete Neilley weather program (yes, I do still
>>>> use that..) I use the following
>>>>
>>>>
>>>>
>>>> DDS|IDS ^S[AP].* .... ([0-3][0-9])([0-2][0-9]) FILE
>>>> /usr/local/ldm/data/weather/(\1:yy)(\1:mm)\1\2.METAR
>>>>
>>>>
>>>>
>>>> Note that it is using the DDMM in the WMO product header (the
>>>> ([0-3][0-9])([0-2][0-9]) part ) to determine the day/hour of the file to
>>>> put it in.
>>>>
>>>>
>>>>
>>>> As we all know, the METAR times are a bit before the actual hour they
>>>> are valid for (e.g. the METAR with a time of 1253 is actually the 13 UTC
>>>> ob, and should be put in the 13 UTC file.
>>>>
>>>>
>>>>
>>>> With the old format (SAUS70/80/90) the time on the WMO header actually
>>>> was the correct time (1300) for the 1253 ob.
>>>>
>>>>
>>>>
>>>> SAUS80 KWBC 271300 RRB^M^M
>>>>
>>>> METAR^M^M
>>>>
>>>>
>>>>
>>>> KMSN 271253Z 23007KT 200V270 10SM FEW180 FEW250 M07/M14 A2991 RMK^M^M
>>>>
>>>> AO2 SLP141 T10721144 $=^M^M
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> However, with the new format (SAUS99) the time on the WMO is generally
>>>> earlier, so in the example here, the 1753 Ob (which should be in the 18 UTC
>>>> file) ends up in the 17 UTC file.
>>>>
>>>>
>>>>
>>>> SAUS99 KWBC 271756 RRH^M^M
>>>>
>>>> METAR^M^M
>>>>
>>>>
>>>>
>>>> KMSN 271753Z 25011G23KT 10SM BKN160 BKN250 04/M12 A2973 RMK AO2^M^M
>>>>
>>>> SLP077 T00391122 10044 21072 58033=^M^M
>>>>
>>>>
>>>>
>>>> I think I found a workaround for my purposes, but wanted to put it out
>>>> there in case it's affecting anyone else..
>>>>
>>>>
>>>>
>>>> Pete
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Pete Pokrandt - System Engineer IV
>>>>
>>>> UW-Madison Dept of Atmospheric and Oceanic Sciences
>>>>
>>>> 608-262-3086 - poker@xxxxxxxxxxxx
>>>> ------------------------------
>>>>
>>>> *From:* ldm-users <ldm-users-bounces@xxxxxxxxxxxxxxxx> on behalf of
>>>> Herzmann, Daryl E [AGRON] <akrherz@xxxxxxxxxxx>
>>>> *Sent:* Monday, January 27, 2025 11:27 AM
>>>> *To:* ldm-users@xxxxxxxxxxxxxxxx <ldm-users@xxxxxxxxxxxxxxxx>
>>>> *Subject:* [ldm-users] The S[AP]US99 update for METARs just happened
>>>>
>>>>
>>>>
>>>> Howdy,
>>>>
>>>> A reminder that this change just happened.
>>>>
>>>> The NWS National Centers for Environmental Prediction (NCEP) Central
>>>> Operations (NCO) creates routine collectives of observational METAR data
>>>> in SAUS70 KWBC, SAUS80 KWBC, SAUS90 KWBC and their counterparts for
>>>> Special observations SPUS70, SPUS80 and SPUS90. These collectives
>>>> require manual intervention to add METAR stations to them. These are
>>>> then distributed over the SBN. Another product, SAUS99 KWBC, and the
>>>> Special observation counterpart SPUS99 KWBC contain any METAR
>>>> observation
>>>> received every minute. Delivering SAUS99 and SPUS99 over the SBN rather
>>>> than the aforementioned collectives will provide more observational data
>>>> and more timely data to users of the SBN.
>>>>
>>>> The replacement with SAUS99 and SPUS99 is expected to occur on or about
>>>> 1200 Coordinated Universal Time (UTC), January 27, 2025. If a Critical
>>>> Weather Day has been issued, this change will be postponed until 1200
>>>> UTC
>>>> on the next non-Critical Weather Day.
>>>>
>>>>
>>>> https://urldefense.com/v3/__https://www.weather.gov/media/notification/pdf_2023_24/scn24-99_add_saus99_to_sbn_aaa.pdf__;!!Mak6IKo!K44CJVSMFQBBR4h9RLswmLpN1JShZZJLM-zvWlrCQT26pq7B-6JV6h9ZIAqtSvktkgbOL-pBQRZTgA9LCC8p$
>>>>
>>>> So far, I have observed 265 SAUS99 products and 114 SPUS99 products
>>>> cross the IDD since about 15 Z
>>>>
>>>> daryl
>>>> _______________________________________________
>>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>>> recorded in the Unidata inquiry tracking system and made publicly
>>>> available through the web. Users who post to any of the lists we
>>>> maintain are reminded to remove any personal information that they
>>>> do not want to be made public.
>>>>
>>>>
>>>> ldm-users mailing list
>>>> ldm-users@xxxxxxxxxxxxxxxx
>>>> For list information or to unsubscribe, visit:
>>>> https://urldefense.com/v3/__https://www.unidata.ucar.edu/mailing_lists/__;!!Mak6IKo!K44CJVSMFQBBR4h9RLswmLpN1JShZZJLM-zvWlrCQT26pq7B-6JV6h9ZIAqtSvktkgbOL-pBQRZTgL9fV5rk$
>>>>
>>>> _______________________________________________
>>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>>> recorded in the Unidata inquiry tracking system and made publicly
>>>> available through the web. Users who post to any of the lists we
>>>> maintain are reminded to remove any personal information that they
>>>> do not want to be made public.
>>>>
>>>>
>>>> ldm-users mailing list
>>>> ldm-users@xxxxxxxxxxxxxxxx
>>>> For list information or to unsubscribe, visit:
>>>> https://www.unidata.ucar.edu/mailing_lists/
>>>>
>>>
>>>
>>> --
>>> - Mike Zuranski
>>> _______________________________________________
>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>> recorded in the Unidata inquiry tracking system and made publicly
>>> available through the web. Users who post to any of the lists we
>>> maintain are reminded to remove any personal information that they
>>> do not want to be made public.
>>>
>>>
>>> ldm-users mailing list
>>> ldm-users@xxxxxxxxxxxxxxxx
>>> For list information or to unsubscribe, visit:
>>> https://www.unidata.ucar.edu/mailing_lists/
>>>
>>
--
- Mike Zuranski
